You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2022/10/07 08:44:39 UTC
[GitHub] [accumulo-testing] keith-turner opened a new issue, #239: Make bulk import random walk verify step print missing markers
keith-turner opened a new issue, #239:
URL: https://github.com/apache/accumulo-testing/issues/239
While debugging apache/accumulo#3006 I spent a while trying to determine what markers were missing. It would be nice to have the verify code just print this information out when it fails. In the test that failed the following error was printed
```
Caused by: java.lang.Exception: Bad key at r00000 cf:000 [] 1665128566432 false=-1
```
Through manual analysis I determined that marker 153 was missing. Would be nice if the error message gave this info, would really speed up debugging issues found by the test. Thinking maybe an error message like the following would be really useful.
```
Caused by: java.lang.Exception: Bad key at r00000 cf:000 [] 1665128566432 false=-1 missing markers : [0000153]
```
The code that prints this error message is here
https://github.com/apache/accumulo-testing/blob/ddc6e589a6703239b7445075172fe88308e5a0ef/src/main/java/org/apache/accumulo/testing/randomwalk/bulk/Verify.java#L73
Below that code is some code that analyzes markers, but it does not run when the count is not zero.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] Manno15 commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1281491633
> If this line was removed, do you think this exception would catch and print out the info you are looking for:
I am currently looking into a solution that is basically that. I believe the other throw exception below will in theory always be thrown if this one is thrown because it compares the marker number.
I haven't been able to reproduce the exception on my end despite the suggestions above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] Manno15 commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1272588641
@keith-turner Did you alter any settings in order to get `Bulk.xml` to fail more consistently?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] Manno15 commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1282194260
That is essentially what I did. Forced it to error at least. Was just
commenting on seeing it naturally.
On Tue, Oct 18, 2022 at 6:33 AM Keith Turner ***@***.***>
wrote:
> I haven't been able to reproduce the exception on my end despite the
> suggestions above.
>
> The problem was intermittent. If you have the fix ***@***.***
> <https://github.com/apache/accumulo/commit/f6c943e70dd14183985faeeefc5b554fcc731fff>
> then you may never see the bug.
>
> For testing purposes, could create a bulk graph locally that will always
> cause verify to fail. I have not tested it, but maybe running the following
> graph would cause verify to always fail quickly. It does a plus one w/o a
> corresponding minus one and then verifies, so the counts should be non-zero.
>
> <module>
>
> <package prefix="bulk" value="org.apache.accumulo.testing.randomwalk.bulk"/>
>
> <init id="bulk.Setup"/>
>
> <node id="bulk.Setup">
> <edge id="bulk.BulkPlusOne" weight="1"/>
> </node>
>
> <node id="bulk.BulkPlusOne">
> <edge id="bulk.Verify" weight="10"/>
> </node>
>
> <node id="bulk.Verify">
> <edge id="END" weight="1"/>
> </node>
>
> </module>
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/accumulo-testing/issues/239#issuecomment-1282176957>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AHASSV2AKYM5UQCFGNKGMKLWDZ4IHANCNFSM6AAAAAAQ7MBQQM>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] DomGarguilo commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1281404976
Also, are you able to reproduce this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] keith-turner commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1296846427
> My current solution is to create a string builder to hold the thrown error message and then actually throw the error later after the marker error would normally get thrown.
That sounds good. Another approach to consider is instead of using the string builder, could log information and then throw an exception with a shorter message after logging. I would go with whatever approach seems easiest to read, but not sure which one is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] DomGarguilo commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1281382026
> The code that prints this error message is here
>
> https://github.com/apache/accumulo-testing/blob/ddc6e589a6703239b7445075172fe88308e5a0ef/src/main/java/org/apache/accumulo/testing/randomwalk/bulk/Verify.java#L73
>
> Below that code is some code that analyzes markers, but it does not run when the count is not zero.
If this line was removed, do you think this exception would catch and print out the info you are looking for:
https://github.com/apache/accumulo-testing/blob/ddc6e589a6703239b7445075172fe88308e5a0ef/src/main/java/org/apache/accumulo/testing/randomwalk/bulk/Verify.java#L93-L95
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] keith-turner commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1273330019
> @keith-turner I can take a look into this if you aren't already.
Yeah go ahead I am not working on it.
> @keith-turner Did you alter any settings in order to get Bulk.xml to fail more consistently?
I made the following tweak to what @milleruntime was doing, running 3 tservers on a single VM setup by uno.
https://github.com/apache/accumulo/issues/2667#issuecomment-1269905344
It may not fail quickly, but leave it running and it will fail. I am seeing multiple different failures when I run this and leave it running for a while. I am running against the following branch of accumulo where I have added some more logging to compaction to see if I can track down some of the issues I am seeing.
https://github.com/keith-turner/accumulo/tree/compaction-logging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] keith-turner commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1282176957
> I haven't been able to reproduce the exception on my end despite the suggestions above.
The problem was intermittent. If you have the fix apache/accumulo@f6c943e70dd14183985faeeefc5b554fcc731fff then you may never see the bug.
For testing purposes, could create a bulk graph locally that will always cause verify to fail. I have not tested it, but maybe running the following graph would cause verify to always fail quickly. It does a plus one w/o a corresponding minus one and then verifies, so the counts should be non-zero.
```xml
<module>
<package prefix="bulk" value="org.apache.accumulo.testing.randomwalk.bulk"/>
<init id="bulk.Setup"/>
<node id="bulk.Setup">
<edge id="bulk.BulkPlusOne" weight="1"/>
</node>
<node id="bulk.BulkPlusOne">
<edge id="bulk.Verify" weight="10"/>
</node>
<node id="bulk.Verify">
<edge id="END" weight="1"/>
</node>
</module>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] keith-turner commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1271316784
This is what the row looks like in accumulo
```
root@uno> table bulk_dev_uk_1088323_1665128094663
root@uno bulk_dev_uk_1088323_1665128094663> scan -r r00000 -np
r00000 cf:000 [] -1
r00000 cf:001 [] -1
r00000 cf:002 [] -1
r00000 cf:003 [] -1
r00000 cf:004 [] -1
r00000 cf:005 [] -1
r00000 cf:006 [] -1
r00000 cf:007 [] -1
r00000 cf:008 [] -1
r00000 cf:009 [] -1
r00000 marker:0000001 [] 1
r00000 marker:0000002 [] 1
r00000 marker:0000003 [] 1
r00000 marker:0000004 [] 1
r00000 marker:0000005 [] 1
r00000 marker:0000006 [] 1
r00000 marker:0000007 [] 1
r00000 marker:0000008 [] 1
r00000 marker:0000009 [] 1
r00000 marker:0000010 [] 1
r00000 marker:0000011 [] 1
r00000 marker:0000012 [] 1
r00000 marker:0000013 [] 1
r00000 marker:0000014 [] 1
r00000 marker:0000015 [] 1
r00000 marker:0000016 [] 1
r00000 marker:0000017 [] 1
r00000 marker:0000018 [] 1
r00000 marker:0000019 [] 1
r00000 marker:0000020 [] 1
r00000 marker:0000021 [] 1
r00000 marker:0000022 [] 1
r00000 marker:0000023 [] 1
r00000 marker:0000024 [] 1
r00000 marker:0000025 [] 1
r00000 marker:0000026 [] 1
r00000 marker:0000027 [] 1
r00000 marker:0000028 [] 1
r00000 marker:0000029 [] 1
r00000 marker:0000030 [] 1
r00000 marker:0000031 [] 1
r00000 marker:0000032 [] 1
r00000 marker:0000033 [] 1
r00000 marker:0000034 [] 1
r00000 marker:0000035 [] 1
r00000 marker:0000036 [] 1
r00000 marker:0000037 [] 1
r00000 marker:0000038 [] 1
r00000 marker:0000039 [] 1
r00000 marker:0000040 [] 1
r00000 marker:0000041 [] 1
r00000 marker:0000042 [] 1
r00000 marker:0000043 [] 1
r00000 marker:0000044 [] 1
r00000 marker:0000045 [] 1
r00000 marker:0000046 [] 1
r00000 marker:0000047 [] 1
r00000 marker:0000048 [] 1
r00000 marker:0000049 [] 1
r00000 marker:0000050 [] 1
r00000 marker:0000051 [] 1
r00000 marker:0000052 [] 1
r00000 marker:0000053 [] 1
r00000 marker:0000054 [] 1
r00000 marker:0000055 [] 1
r00000 marker:0000056 [] 1
r00000 marker:0000057 [] 1
r00000 marker:0000058 [] 1
r00000 marker:0000059 [] 1
r00000 marker:0000060 [] 1
r00000 marker:0000061 [] 1
r00000 marker:0000062 [] 1
r00000 marker:0000063 [] 1
r00000 marker:0000064 [] 1
r00000 marker:0000065 [] 1
r00000 marker:0000066 [] 1
r00000 marker:0000067 [] 1
r00000 marker:0000068 [] 1
r00000 marker:0000069 [] 1
r00000 marker:0000070 [] 1
r00000 marker:0000071 [] 1
r00000 marker:0000072 [] 1
r00000 marker:0000073 [] 1
r00000 marker:0000074 [] 1
r00000 marker:0000075 [] 1
r00000 marker:0000076 [] 1
r00000 marker:0000077 [] 1
r00000 marker:0000078 [] 1
r00000 marker:0000079 [] 1
r00000 marker:0000080 [] 1
r00000 marker:0000081 [] 1
r00000 marker:0000082 [] 1
r00000 marker:0000083 [] 1
r00000 marker:0000084 [] 1
r00000 marker:0000085 [] 1
r00000 marker:0000086 [] 1
r00000 marker:0000087 [] 1
r00000 marker:0000088 [] 1
r00000 marker:0000089 [] 1
r00000 marker:0000090 [] 1
r00000 marker:0000091 [] 1
r00000 marker:0000092 [] 1
r00000 marker:0000093 [] 1
r00000 marker:0000094 [] 1
r00000 marker:0000095 [] 1
r00000 marker:0000096 [] 1
r00000 marker:0000097 [] 1
r00000 marker:0000098 [] 1
r00000 marker:0000099 [] 1
r00000 marker:0000100 [] 1
r00000 marker:0000101 [] 1
r00000 marker:0000102 [] 1
r00000 marker:0000103 [] 1
r00000 marker:0000104 [] 1
r00000 marker:0000105 [] 1
r00000 marker:0000106 [] 1
r00000 marker:0000107 [] 1
r00000 marker:0000108 [] 1
r00000 marker:0000109 [] 1
r00000 marker:0000110 [] 1
r00000 marker:0000111 [] 1
r00000 marker:0000112 [] 1
r00000 marker:0000113 [] 1
r00000 marker:0000114 [] 1
r00000 marker:0000115 [] 1
r00000 marker:0000116 [] 1
r00000 marker:0000117 [] 1
r00000 marker:0000118 [] 1
r00000 marker:0000119 [] 1
r00000 marker:0000120 [] 1
r00000 marker:0000121 [] 1
r00000 marker:0000122 [] 1
r00000 marker:0000123 [] 1
r00000 marker:0000124 [] 1
r00000 marker:0000125 [] 1
r00000 marker:0000126 [] 1
r00000 marker:0000127 [] 1
r00000 marker:0000128 [] 1
r00000 marker:0000129 [] 1
r00000 marker:0000130 [] 1
r00000 marker:0000131 [] 1
r00000 marker:0000132 [] 1
r00000 marker:0000133 [] 1
r00000 marker:0000134 [] 1
r00000 marker:0000135 [] 1
r00000 marker:0000136 [] 1
r00000 marker:0000137 [] 1
r00000 marker:0000138 [] 1
r00000 marker:0000139 [] 1
r00000 marker:0000140 [] 1
r00000 marker:0000141 [] 1
r00000 marker:0000142 [] 1
r00000 marker:0000143 [] 1
r00000 marker:0000144 [] 1
r00000 marker:0000145 [] 1
r00000 marker:0000146 [] 1
r00000 marker:0000147 [] 1
r00000 marker:0000148 [] 1
r00000 marker:0000149 [] 1
r00000 marker:0000150 [] 1
r00000 marker:0000151 [] 1
r00000 marker:0000152 [] 1
r00000 marker:0000154 [] 1
r00000 marker:0000155 [] 1
r00000 marker:0000156 [] 1
r00000 marker:0000157 [] 1
r00000 marker:0000158 [] 1
r00000 marker:0000159 [] 1
r00000 marker:0000160 [] 1
r00000 marker:0000161 [] 1
r00000 marker:0000162 [] 1
r00000 marker:0000163 [] 1
r00000 marker:0000164 [] 1
r00000 marker:0000165 [] 1
r00000 marker:0000166 [] 1
r00000 marker:0000167 [] 1
r00000 marker:0000168 [] 1
r00000 marker:0000169 [] 1
r00000 marker:0000170 [] 1
r00000 marker:0000171 [] 1
r00000 marker:0000172 [] 1
r00000 marker:0000173 [] 1
r00000 marker:0000174 [] 1
r00000 marker:0000175 [] 1
r00000 marker:0000176 [] 1
r00000 marker:0000177 [] 1
r00000 marker:0000178 [] 1
r00000 marker:0000179 [] 1
r00000 marker:0000180 [] 1
r00000 marker:0000181 [] 1
r00000 marker:0000182 [] 1
r00000 marker:0000183 [] 1
r00000 marker:0000184 [] 1
r00000 marker:0000185 [] 1
r00000 marker:0000186 [] 1
r00000 marker:0000187 [] 1
r00000 marker:0000188 [] 1
r00000 marker:0000189 [] 1
r00000 marker:0000190 [] 1
r00000 marker:0000191 [] 1
r00000 marker:0000192 [] 1
r00000 marker:0000193 [] 1
r00000 marker:0000194 [] 1
r00000 marker:0000195 [] 1
r00000 marker:0000196 [] 1
r00000 marker:0000197 [] 1
r00000 marker:0000198 [] 1
r00000 marker:0000199 [] 1
r00000 marker:0000200 [] 1
r00000 marker:0000201 [] 1
r00000 marker:0000202 [] 1
r00000 marker:0000203 [] 1
r00000 marker:0000204 [] 1
r00000 marker:0000205 [] 1
r00000 marker:0000206 [] 1
r00000 marker:0000207 [] 1
r00000 marker:0000208 [] 1
r00000 marker:0000209 [] 1
r00000 marker:0000210 [] 1
r00000 marker:0000211 [] 1
r00000 marker:0000212 [] 1
r00000 marker:0000213 [] 1
r00000 marker:0000214 [] 1
r00000 marker:0000215 [] 1
r00000 marker:0000216 [] 1
r00000 marker:0000217 [] 1
r00000 marker:0000218 [] 1
r00000 marker:0000219 [] 1
r00000 marker:0000220 [] 1
r00000 marker:0000221 [] 1
r00000 marker:0000222 [] 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] Manno15 commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1272367603
@keith-turner I can take a look into this if you aren't already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] Manno15 commented on issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #239:
URL: https://github.com/apache/accumulo-testing/issues/239#issuecomment-1296284235
My current solution is to create a string builder to hold the thrown error message and then actually throw the error later after the marker error would normally get thrown. That way, the user would have the most complete information set. The initial thrown error for a bad key would cause the errors below to trigger since it is dependent on the marker number (which one would be missing). Thoughts on this approach @keith-turner @DomGarguilo? Another approach would do the row iterator first but I assume similar issues would arise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-testing] Manno15 closed issue #239: Make bulk import random walk verify step print missing markers
Posted by GitBox <gi...@apache.org>.
Manno15 closed issue #239: Make bulk import random walk verify step print missing markers
URL: https://github.com/apache/accumulo-testing/issues/239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org