You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by "paulirwin (via GitHub)" <gi...@apache.org> on 2024/02/25 15:58:10 UTC
[I] Fix test name reporting when test is in a base class [lucenenet]
paulirwin opened a new issue, #923:
URL: https://github.com/apache/lucenenet/issues/923
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Task description
Context: https://github.com/apache/lucenenet/pull/914#discussion_r1494919429
From @NightOwl888:
> we should have a new issue about fixing test reports so the correct superclass is reported for tests that are declared in a base class.
Also see: https://github.com/apache/lucenenet/pull/914#discussion_r1494864427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix test name reporting when test is in a base class [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin commented on issue #923:
URL: https://github.com/apache/lucenenet/issues/923#issuecomment-2568711978
Yeah, I had that working locally before I found the solution. But here's the PR: https://github.com/apache/lucenenet/pull/1086
That is running on ADO now, passes in GitHub. You can see in the GitHub test output that it uses the full name of the test including the derived class name (not the base class). Let me know if that causes any problems in VS for you locally if it picks up the runsettings file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix test name reporting when test is in a base class [lucenenet]
Posted by "NightOwl888 (via GitHub)" <gi...@apache.org>.
NightOwl888 commented on issue #923:
URL: https://github.com/apache/lucenenet/issues/923#issuecomment-2568639820
Well, at least the analyzer will help prevent developers from backing out the overrides. At one point, these were overridden to fix all of the names, but it was commented and considered a temporary workaround. There have been at least 2 developers that decided they were "helping" by removing them, though. And now they are mostly gone.
Do note that this pertains to tests that inherit any abstract class that contain tests. Some of them may be classes that are 2 or more levels of inheritance from the base class they need to override. The base class that contains the test does not necessarily need to be abstract.
It seems more like we should either change the name of the test or report this to NUnit so they can provide a fix or another workaround. It would be much simpler for us and our users to maintain if the inheritance-based test framework understood how to self report its own inherited tests.
It may also be possible to come up with a solution based on how xunit reports them. I don't ever recall having this issue with xunit even though J2N uses several tests that are inherited this way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix test name reporting when test is in a base class [lucenenet]
Posted by "NightOwl888 (via GitHub)" <gi...@apache.org>.
NightOwl888 commented on issue #923:
URL: https://github.com/apache/lucenenet/issues/923#issuecomment-2571250855
Just for reference, the documentation on how to supply runsettings on the command is at: https://github.com/Microsoft/vstest-docs/blob/main/docs/RunSettingsArguments.md. For CI purposes, it is important to note that the syntax varies by shell. We explicitly specify bash as the shell so the command will work on every OS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix test name reporting when test is in a base class [lucenenet]
Posted by "NightOwl888 (via GitHub)" <gi...@apache.org>.
NightOwl888 commented on issue #923:
URL: https://github.com/apache/lucenenet/issues/923#issuecomment-2568702773
Come to think of it, the only thing we really need is for the test message to have the name of the test subclass in it when there is a failure. A simple way to accomplish that would be to add the name of the class to the message in the [`LuceneTestCase.TearDown()` method](https://github.com/apache/lucenenet/blob/dbc602c7f872189516dc132045f1cc2b69d5ca7e/src/Lucene.Net.TestFramework/Util/LuceneTestCase.cs#L941-L976) by doing a Reflection call (`GetType().FullName`) at that point (since it only prints if the test fails).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix test name reporting when test is in a base class [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin commented on issue #923:
URL: https://github.com/apache/lucenenet/issues/923#issuecomment-2569939849
PR #1086 has been updated. Just FYI in case someone discovers this issue later looking for a solution, if you pass `NUnit.DisplayName=FullName` after the `--` separator in `dotnet test` (i.e. `dotnet test {any regular args here} -- NUnit.DisplayName=FullName`), it will work without having a runsettings file. Alternatively, you can set it in a runsettings file if you'd prefer:
```xml
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<NUnit>
<DisplayName>FullName</DisplayName>
</NUnit>
</RunSettings>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix test name reporting when test is in a base class [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin commented on issue #923:
URL: https://github.com/apache/lucenenet/issues/923#issuecomment-2568530481
I looked into this yesterday, and the only place where the test class name seems to be "reported" is in the stack trace. I don't believe we can easily manipulate the stack traces to include the child class name.
We could look into modifying the NUnit test naming pattern, although they [strongly discourage modifying that](https://docs.nunit.org/articles/nunit/running-tests/Template-Based-Test-Naming.html). I will give it a try, though.
If that doesn't work or causes issues with the IDE, I think perhaps our best option, per our other discussions about having an internal analyzer project, could be to create an analyzer with quick fix that looks for tests defined in a base class that haven't been overridden in the derived class, and have the quick fix scaffold out the overrides. While this wouldn't solve the issue as originally defined, it would help solve the "doesn't scale well" concern by automating the detection of missing overrides in the IDE and build, and the fix in the IDE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix test name reporting when test is in a base class [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin closed issue #923: Fix test name reporting when test is in a base class
URL: https://github.com/apache/lucenenet/issues/923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org