You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2020/07/01 17:47:19 UTC
[GitHub] [accumulo] milleruntime opened a new issue #1647: Compaction tests failing in TableOperationsIT
milleruntime opened a new issue #1647:
URL: https://github.com/apache/accumulo/issues/1647
TableOperationsIT is now failling with the merge of #1605. I am seeing 3 tests failing:
testCompactEmptyTableWithGeneratorIterator
testCompactEmptyTableWithGeneratorIterator_Splits_Partial
testCompactEmptyTableWithGeneratorIterator_Splits
They have similar assertion errors:
```
java.lang.AssertionError: expected:<{a1 colF3:colQ3 [] 1593624908123 false=1, c1 colF3:colQ3 [] 1593624908123 false=1, m1 colF3:colQ3 [] 1593624908123 false=1}> but was:<{}>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.apache.accumulo.test.TableOperationsIT.testCompactEmptyTableWithGeneratorIterator(TableOperationsIT.java:288)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:834)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-652664533
I did not see this issue an opened #1648
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii closed issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
ctubbsii closed issue #1647:
URL: https://github.com/apache/accumulo/issues/1647
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653024418
With the limited full time contributors we have now, I would error on the side of less complexity by removing the test. With this feature not advertised, it is possible, even likely, that it is not used. As with all new features, there is new behavior so users have a different set of expectations. We can always add the complexity later if there is a legitimate use case. I would remove the ```@Test``` annotation with a comment, citing the issue number with your comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-652665260
It seems sensible to me that "generator compactions" would be allowed to run. However, I have no particular opinion on how they should be scheduled. From the user's perspective, if I run a compaction, with a generator iterator configured in a compaction scope, I would expect the resulting table to contain the generated data when the compaction is complete.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653278435
I was thought of a fairly simple way to implement empty compactions w/o riddling the code w/ special cases for the empty set. Using a set with a special URI for this case instead of implementing special handling for the empty set made the changes quite small. I made the changes in the following branch and the test pass now.
https://github.com/keith-turner/accumulo/tree/accumulo-1647
These changes build on the changes in #1649.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner edited a comment on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner edited a comment on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-652664533
I did not see this issue and opened #1648. I closed #1648 as a dup, but it would still be good to look at the description. I am uncertain what to do. Should I remove the test or add the complexity to support empty compactions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-1069681295
> Should this ticket be closed/renamed? The referenced failing tests were removed in [aef266c](https://github.com/apache/accumulo/commit/aef266c31c8a0dfbec163135c5a21606aa038344).
Created #2570
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
milleruntime commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653182711
From @EdColeman
Now sure if this helps - but there is a use case for having an empty tablet. If a user processes a data set and generates multiple tables, say data, metrics and errors - the ideal case would be that there are no errors (so an empty table). Rather than handling the logic of if the table does not exist, there must be no errors, it was simpler for their query / scan logic to have the empty table. The idea is that if there are issues, they could scan the errors to determine where things are going off the rails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner closed issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner closed issue #1647:
URL: https://github.com/apache/accumulo/issues/1647
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653234692
> My main concern is inconsistent behavior. Iterators that generate data will work when there is existing files... but not work when there isn't.
That is a good point. When I was thinking about this, I thought the generator iterators would only generate data when the source iterator was empty, but I suppose that does not have to be the case. If a tablet has data, then a generator iterator would have to merge the generated data with existing data while respecting sorting. Also the generator iterator would need to only generate data within the bounds of the table. Actually writing one of these that is correct seems non-trivial.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner edited a comment on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner edited a comment on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653234692
> My main concern is inconsistent behavior. Iterators that generate data will work when there is existing files... but not work when there isn't.
That is a good point. When I was thinking about this, I thought the generator iterators would only generate data when the source iterator was empty, but I suppose that does not have to be the case. If a tablet has data, then a generator iterator would have to merge the generated data with existing data while respecting sorting. Also the generator iterator would need to only generate data within the bounds of the tablet. Actually writing one of these that is correct seems non-trivial.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-953072783
Should this ticket be closed/renamed? The referenced failing tests were removed in aef266c31c8a0dfbec163135c5a21606aa038344.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653226825
My main concern is inconsistent behavior. Iterators that generate data will work when there is existing files... but not work when there isn't. This is very unintuitive, and the actual state that the behavior turns on is invisible to the end user who executes a compaction.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner edited a comment on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner edited a comment on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-652664533
I did not see this issue and opened #1648
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner edited a comment on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
keith-turner edited a comment on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653278435
I thought of a fairly simple way to implement empty compactions w/o riddling the code w/ special cases for the empty set. Using a set with a special URI for this case instead of implementing special handling for the empty set made the changes quite small. I made the changes in the following branch and the test pass now.
https://github.com/keith-turner/accumulo/tree/accumulo-1647
These changes build on the changes in #1649.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime edited a comment on issue #1647: Compaction tests failing in TableOperationsIT
Posted by GitBox <gi...@apache.org>.
milleruntime edited a comment on issue #1647:
URL: https://github.com/apache/accumulo/issues/1647#issuecomment-653182711
From @EdColeman
Not sure if this helps - but there is a use case for having an empty tablet. If a user processes a data set and generates multiple tables, say data, metrics and errors - the ideal case would be that there are no errors (so an empty table). Rather than handling the logic of if the table does not exist, there must be no errors, it was simpler for their query / scan logic to have the empty table. The idea is that if there are issues, they could scan the errors to determine where things are going off the rails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org