Repository navigation
Moved benchmark code to NUnit.InternalTools repo, and removed it from… - #5203
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Removes the nunit.framework.benchmarks project from the NUnit framework repo/solution, consistent with moving benchmark tooling to a separate repository (Fixes #5198).
Changes:
- Deleted the
nunit.framework.benchmarksproject file and all benchmark source files. - Removed the benchmarks project entry from
nunit.slnx.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/NUnitFramework/benchmarks/nunit.framework.benchmarks/nunit.framework.benchmarks.csproj | Removes the BenchmarkDotNet-based benchmarks project definition. |
| src/NUnitFramework/benchmarks/nunit.framework.benchmarks/Program.cs | Removes the benchmarks entry point / BenchmarkSwitcher wiring. |
| src/NUnitFramework/benchmarks/nunit.framework.benchmarks/StreamsComparerBenchmark.cs | Removes stream comparison benchmark implementations. |
| src/NUnitFramework/benchmarks/nunit.framework.benchmarks/ParamAttributeTypeConversionsBenchmark.cs | Removes param conversion benchmarks and related helper implementations. |
| src/NUnitFramework/benchmarks/nunit.framework.benchmarks/FrameworkControllerBenchmark.cs | Removes framework controller performance benchmarks. |
| src/NUnitFramework/benchmarks/nunit.framework.benchmarks/CompositeWorkItemBenchmark.cs | Removes CompositeWorkItem/test action performance benchmark. |
| src/NUnitFramework/benchmarks/nunit.framework.benchmarks/ByteAllocationBenchmark.cs | Removes byte allocation/pooling benchmarks. |
| nunit.slnx | Removes the benchmarks project from the solution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
Author
|
@stevenaw Can you have a quick look ? |
stevenaw
approved these changes
Mar 30, 2026
stevenaw
left a comment
Member
There was a problem hiding this comment.
LGTM. Project and all references seem captured.
Thanks @OsirisTerje
OsirisTerje
added a commit
that referenced
this pull request
May 19, 2026
* Exclude null from the "Special handling when sole argument is an object[]" * Change the test to reflect the new behaviour * Exclude null from the "Special handling when sole argument is an object[]" * Change the test to reflect the new behaviour * Update Cake.Tool from 4.0.0 to 5.1.0 Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Agent-Logs-Url: https://github.com/nunit/nunit/sessions/eff1c2d0-f67d-4b74-995e-4e0592f2ec27 * Update Cake.Tool to 5.1.0 in .config-macos/dotnet-tools.json Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Agent-Logs-Url: https://github.com/nunit/nunit/sessions/b5ea6ea3-eed1-40bd-924e-f4822f0803fc * Add `mainthread` argument parsing to nunitlite * Update src/NUnitFramework/framework/Attributes/PlatformNames.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix comments * Fix with GetEffectiveProperty * Fix syntax in IncludeExcludeAttribute * Remove unused using * Restore baseclass checking that the constraint expects string arguments. * Clarify intent by replacing anonymous `new` with `new ResultState` * Add SameAs<T> to constraintexpression * Do not unpack an array if it is generic or assignable from IEnumerable<> * Don't unpack array if the target method only expects a single parameter. * Update Cake to version 6.1 (#5201) * Bump cake versions to 6.1 * Update our cakescripts project to match the runtime of the current cake version * Updated minver, made it match with the adapter code, and get rid of the warning. * Fix minver version on std --------- Co-authored-by: Terje Sandstrom <terje@hermit.no> * Optimize the unpack method and add some edge case test coverage for existing behaviour. * Ensure the same exception is thrown today as before * Update partition filter to use our GC polyfill * Update src/NUnitFramework/framework/Internal/Extensions/ArgumentsExtensions.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add test for copilot-identified feedback * Moved benchmark code to NUnit.InternalTools repo, and removed it from… (#5203) * Moved benchmark code to NUnit.InternalTools repo, and removed it from framework * Fixing copilot review comments, removing benchmark related stuff in some other files. * Updated third party dev packages licence and versions * Add test coverage for platform attribute behaviours * Respond to feedback * Use non-standard names to augment PlatformHelper. That way it won't affect the standard operation * Add real os specific tests * Add examples of test with Excludes * Fixes #5211 * Add missing invalid depth guard and test coverage of generic configuration * Changed to more informative exception type * Review changes * Exposed the last Recorded Exception in TestResult (#5217) * Exposed the last Recorded Exception in TestResult * Update src/NUnitFramework/tests/TestContextTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Display values of properties in case of an unexpected Exception (#5216) * Display values of properties in case of an unexpected Exception * Cache Exception properties and don't add overridden base class properties * Phase 1 - moving towards dotnet build (#5210) * Phase 1 - moving towards dotnet build * Update src/NUnitFramework/Directory.Build.props Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removed Version property * fixed unreadable logs (blue color) and added copilot deletions to the target file * updated minver * Changed to correct sdk for building, kept an error to check the output * Updated yml * adding a compiler error * fix compiler error again * Adding caching * remove caching * Updated tests that were flaky --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Phase 2 - Issue5207 simplify testing and building in cake (#5213) * Phase 1 - moving towards dotnet build * Update src/NUnitFramework/Directory.Build.props Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removed Version property * fixed unreadable logs (blue color) and added copilot deletions to the target file * updated minver * Fixing cake builds and converting testing * fixing action workflows * fixed copilot comments * fixed copilot comments * increased time for LargeStringCollectionsInReversedOrder * increased time for LargeStringCollectionsInSameOrder * increased time for LargeStringCollectionsInSameOrder * removed SA1001, it crashes with VS formatting * Increased timeout so that CI builds works, they run fast locally * removed windows runs from linux and mac * removed windows runs from linux and mac * reducing CI noise * reducing CI noise * fix flaky windows test * fix other flaky windows tests * Revert "removed SA1001, it crashes with VS formatting" This reverts commit 6547a48. * Fix so that VS and SA agree on formatting. * Changed to correct sdk for building, kept an error to check the output * Updated yml * adding a compiler error * fix compiler error again * Adding caching * Adding trx reporting * removed cache again * Changed to Dorny due to container usage of enrico * changed to minimal * Changed failed on error * trying to get linux/mac to work too * Fixed error in reporting * Getting back a bit more than quiet * adjusting verbosity * changed output from tests * reverted change to test that didnt actually work, it might still be flaky * Updated Ci workflow * updated workflow * finickling the workflow again * converted to python and fixed paths * adding debugging * converted to python and fixed paths * adding debugging 2 * test became wrong, now fails correctly * fixing linux/mac * fixing stacktrace * fixing header * fixing stacktrace * Set the delibertaelyfailing test back to Explicit * Fix null reference warning in TestResultsParser * Remove netstandard/netcoreapp from FrameworkPattern Drop IgnoreCase and ToLower as frameworks are case-sensitive. We also only use lower case consistently. * Making flaky test explicit * Reverted TextOutputTests * UPdated building instructions * Updated adapter to 6.2 for all but the net6 which uses 5.2 * Tried to make the DelayedConstraintTests a bit less flaky * New attempt to make it work on linux/mac, * And again * Set TestingPlatformDotnetTestSupport=true, and simplified conditionals * and again, MTP is a mess * Reverted back to adapter 5.2.0 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Issue4824 replace delegates (#5196) * Remove Obsolute Apply(ref TActual) * Add overloads for Action and Func<Task> This is meant to replace TestDelegate and AsyncTestDelegate respectively. To prevent overload resolution conflicts with Lambda, the old methods are deprioritized * Add overloads for Func<TActual> * Add NUnit.Analyzer suppressions for Func<T> Revert when nunit/nunit.analyzers#982 is released * Mark Delegate declarations as Obsolete * Remove some test usages of delegates * Fix merge conflicts after rebase * Fix merge conflicts after rebase --------- Co-authored-by: Terje Sandstrom <terje@hermit.no> * Fastpaths for equivalency + super/sub set tests for certain types (#5224) * Add generic collectiontally and supported classes. * Move the tally fastpath into a base class so that sub/super sets can use it too * A bit of cleanup before trying to fix O(n^2) impl * O(n+m) optimization * Fastpath for bytes * Add "unsorted" dictionary-based path. 5x faster for strings * Add copilot instruction to preserve performance * Convert CollectionTally{T} to use strategy pattern internally with different paths for primitives or others * Lazily instantiate the strategies * A bit of usability cleanup for CollectionTally{T} * Make the generic collectiontally{t} work for types not explicitly targeted * Fix a few bugs * Better encapsulate the state management required by the collection tally strategies and ensure all paths go to new collection tally class * Mark the public protected method obsolete * Use AssignableFrom for sortable check since it's available on all runtimes and still intrinsic * Update the collectiontally tests to target the generic version * Lower the largecollection timing thresholds by factor of 5 since the perf improvements seem to speed it up by at least factor of 6 * Re-add stringcollection special case as it previously existed in non-generic CollectionTally * Add test coverage for generic IEnumerable<T>.IsSortable() method and for some additional collectiontally paths * Update src/NUnitFramework/framework/Constraints/CollectionTally{T}.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/NUnitFramework/tests/Constraints/CollectionEquivalentConstraintTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/NUnitFramework/framework/Constraints/CollectionTally{T}.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/NUnitFramework/tests/Internal/Extensions/IEnumerableExtensionTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Improve sort-related fastpath inference not types not explicity targeted * Fix some outstanding bugs when dealing with object-based collections to maximize the chosen strategy * Make some new things internal instead of public. Make it so tuples + valuetuples can use more efficient strategies if their underlying types support it * Allow the NUnit Comparer adapter to adhere to hashcode/equality semantics * Update src/NUnitFramework/framework/Constraints/NUnitEqualityComparer.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Rename HashCodeExtensions.cs to HashCode.cs to match the contained type Agent-Logs-Url: https://github.com/nunit/nunit/sessions/f62a73e3-7ab5-4940-8412-0921b378452c Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> * Remove the need to base a base class field to a base class method. * Fix strangling between CollectionTally and ItemsStrategy This by moving fields from the CollectionTally class to the ItemsStrategy class. * Update src/NUnitFramework/tests/Constraints/CollectionEquivalentConstraintTests.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Revert "Fix strangling between CollectionTally and ItemsStrategy" This reverts commit b9b7e90. * Remove tolerance from the equality adapter for now * Make arguments consistent in test helper * Move instantiation back to a SetUp callback * Remove unneccesary check on _comparePropertiesConfiguration * Add descriptive comment for why to target a few explicit type fast-paths * Re-add descriptive elapsed time for the equivalency performance tests * Rename "MergeSortItemsStrategy" to "SortAndScanItemsStrategy" for better clarity * Remove null default arg. It's unused * Add test for nulls * Explicitly allow for nullable missing/extra items * Avoid accessing private variables from both class and inner class --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Reorganize collectionequiavelncy tests so that only performance-focused ones are excluded from MacOS. * Enable collectionequivalency tests on MacOS * Make TestParameters.ArgDisplayNames getter public, #3822 (#5236) * Adjust a few cases for passing exceptions to AfterTest hook * Add coverage for capturing exception for setup/teardown * After exception-related test cases for hooks and test actions * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove unused result variables to fix CS0219 warnings-as-errors build failure Agent-Logs-Url: https://github.com/nunit/nunit/sessions/cf703463-aadb-4e1b-9dfc-e810ffaba573 Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> * Ensure the approach to testing path constraints is consistent across OSes and sub-constraint * chore: Bump NUnit.Analyzers and remove TODOs (#5248) * fix RetryAttribute * add unit test * improve unit test * Apply suggestions from code review Co-authored-by: Terje Sandstrom <terje@hermit.no> * fix `RetryOnAllowedStopsOnOtherException` and add `using (Assert.EnterMultipleScope())` to applicable tests in `RetryAttributeTests` * apply suggestions * Refactor DateTimeOffsetEquality tests to use explicit [Test] methods to avoid invalid cases being generated from data points * Reduce skipped count by filtering out invalid test cases for runtimeframework ParseByClrVersion tests * Remove tests duplicated by DatapointTests.WorksOnField * Move the nullvaluesource tests into testdata so that we can remove the "explicit" usage on the original test method and prevent them reporting as "skipped" * Respond to most feedback * Merged with main, and fixed #5253 * Fixed some comments * Apply suggestion from @stevenaw * Apply suggestions from code review Co-authored-by: Steven Weerdenburg <stevenaw@users.noreply.github.com> * Fix up a few things from the in-browser suggestions. It all passed locally * Update src/NUnitFramework/tests/AsyncExecutionApiAdapter.Constraint-based.cs Co-authored-by: Steven Weerdenburg <stevenaw@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Fixed comments * Fix comment * Fix nullanble issue --------- Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Co-authored-by: stevenaw <sweerdenburg@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Steven Weerdenburg <stevenaw@users.noreply.github.com> Co-authored-by: Will Dean <will.dean@indcomp.co.uk> Co-authored-by: Paul Irwin <paulirwin@gmail.com> Co-authored-by: Mikkel Nylander Bundgaard <mikkelbu@users.noreply.github.com> Co-authored-by: ackhack <christoph.fuerbacher@gmail.com> Co-authored-by: Christoph Fürbacher <43985368+ackhack@users.noreply.github.com>
OsirisTerje
added a commit
that referenced
this pull request
May 24, 2026
* Remove the non-generic sameas methods and constraint * feat: Drop .NET 6 as a build target of the framework * TestCaseData rework (#5138) * Reworked the TestCaseData.cs type into a hierarchy with generic types providing type safety without code duplication - Added type TestCaseDataBase.cs to be the new root of the tree of types for test case data. Migrated all Fluent Access Modifiers on TestCaseData except for Returns here. - Added type TestCaseDataWithReturnBase.cs to extend TestCaseDataBase with return values. Migrated the Returns Fluent Access Modifier here. - Reworked the non-generic TestCaseData class in TestCaseData.cs as a subclass of TestCaseDataWithReturnBase<TestCaseData, object>. Added an implicit conversion operator to TestCaseDataBase<TSelf> so that existing code that assumes any derived TestCaseData can be assigned to TestCaseData still works. - Reworked the generic TestCaseData<T...> classes to inherit from TestCaseDataBase<...>. - Added class TestCaseDataWithReturn.cs that provides subclasses of TestCaseDataWithReturn.cs for up to 5 generic argumnents. - Updated IgnoredTestCaseData.cs to use TestCaseParameters as the common non-generic root type for all test case data, since TestCaseData is now no longer an ancestor. - Added automated testing of TestCaseData<T...> and TestCaseDataWithReturn<T..., TReturn>. * Added missing constructors in TestCaseDataBase.cs and TestCaseDataWithReturnBase.cs to allow cloning from another TestCaseParameters object. * Added static Create methods to TestCaseData.cs. Removed unnecessary constructors from TestCaseDataBase.cs and TestCaseDataWithReturnBase.cs. Removed constructors accepting expectedReturnValue from each of the TestCaseDataWithReturn generic types in TestCaseDataWithReturn.cs. Removed associated tests from TestCaseSourceTests.cs. Added a static Return method to each of the TestCaseData generic types in TestCaseData.cs which transform the instance to a TestCaseDataWithReturn<.., TReturn>. Added an internal constructor to each of the TestCaseDataWithReturn generic types to import data from any object of a type deriving from TestCaseParameters. Added tests in TestCaseDataTests.cs that verify (at compile time) that TestCaseData.Create<..> and TestCaseData<..>.Returns<TReturn> yield a reference compatible with the expected type and that the migration of data from TestCaseData<..> to TestCaseDataWithReturn<.., TReturn> works properly. * Showcase the use of the Create factory method alongside the constructor * Added test cases for TestCaseDataWithReturn<T> with 2 and 3 arguments --------- Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Update src/NUnitFramework/framework/Attributes/OSPlatformConverter.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * chore: Update platform constants * `#if NET8_0_OR_GREATER` => `+#if !NETFRAMEWORK` * `#if !NET8_0_OR_GREATER` => `#if NETFRAMEWORK` I've kept the explanation and the constants in BUILDING.md * Fix merge issue * Change namespace for string, collection, file and directory classes * Remove the non-generic sameas methods and constraint * TestCaseData rework (#5138) * Reworked the TestCaseData.cs type into a hierarchy with generic types providing type safety without code duplication - Added type TestCaseDataBase.cs to be the new root of the tree of types for test case data. Migrated all Fluent Access Modifiers on TestCaseData except for Returns here. - Added type TestCaseDataWithReturnBase.cs to extend TestCaseDataBase with return values. Migrated the Returns Fluent Access Modifier here. - Reworked the non-generic TestCaseData class in TestCaseData.cs as a subclass of TestCaseDataWithReturnBase<TestCaseData, object>. Added an implicit conversion operator to TestCaseDataBase<TSelf> so that existing code that assumes any derived TestCaseData can be assigned to TestCaseData still works. - Reworked the generic TestCaseData<T...> classes to inherit from TestCaseDataBase<...>. - Added class TestCaseDataWithReturn.cs that provides subclasses of TestCaseDataWithReturn.cs for up to 5 generic argumnents. - Updated IgnoredTestCaseData.cs to use TestCaseParameters as the common non-generic root type for all test case data, since TestCaseData is now no longer an ancestor. - Added automated testing of TestCaseData<T...> and TestCaseDataWithReturn<T..., TReturn>. * Added missing constructors in TestCaseDataBase.cs and TestCaseDataWithReturnBase.cs to allow cloning from another TestCaseParameters object. * Added static Create methods to TestCaseData.cs. Removed unnecessary constructors from TestCaseDataBase.cs and TestCaseDataWithReturnBase.cs. Removed constructors accepting expectedReturnValue from each of the TestCaseDataWithReturn generic types in TestCaseDataWithReturn.cs. Removed associated tests from TestCaseSourceTests.cs. Added a static Return method to each of the TestCaseData generic types in TestCaseData.cs which transform the instance to a TestCaseDataWithReturn<.., TReturn>. Added an internal constructor to each of the TestCaseDataWithReturn generic types to import data from any object of a type deriving from TestCaseParameters. Added tests in TestCaseDataTests.cs that verify (at compile time) that TestCaseData.Create<..> and TestCaseData<..>.Returns<TReturn> yield a reference compatible with the expected type and that the migration of data from TestCaseData<..> to TestCaseDataWithReturn<.., TReturn> works properly. * Showcase the use of the Create factory method alongside the constructor * Added test cases for TestCaseDataWithReturn<T> with 2 and 3 arguments --------- Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * feat: Drop .NET 6 as a build target of the framework * Update src/NUnitFramework/framework/Attributes/OSPlatformConverter.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * chore: Update platform constants * `#if NET8_0_OR_GREATER` => `+#if !NETFRAMEWORK` * `#if !NET8_0_OR_GREATER` => `#if NETFRAMEWORK` I've kept the explanation and the constants in BUILDING.md * Change namespace for string, collection, file and directory classes * Add generic type constraints * Add tests and missing xmldoc * Add throwsconstraint tests * Update src/NUnitFramework/framework/Assert.Exceptions.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Update src/NUnitFramework/framework/Assert.Exceptions.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Update src/NUnitFramework/framework/Assert.Exceptions.Async.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Update src/NUnitFramework/framework/Assert.Exceptions.Async.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Restore binary-compliant return type * Restructure some of the inheritance hierarchy to have all generic type constraints under a common generic TypeContraint<T> * Re-add tests and adjust existing to account for new inheritance hierarchy * Completely separate inheritance hierarchy for generic type-checking constraints * Add quick paramname validation to existing test * Release5x mainmerge issue5253 (#5262) * Exclude null from the "Special handling when sole argument is an object[]" * Change the test to reflect the new behaviour * Exclude null from the "Special handling when sole argument is an object[]" * Change the test to reflect the new behaviour * Update Cake.Tool from 4.0.0 to 5.1.0 Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Agent-Logs-Url: https://github.com/nunit/nunit/sessions/eff1c2d0-f67d-4b74-995e-4e0592f2ec27 * Update Cake.Tool to 5.1.0 in .config-macos/dotnet-tools.json Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Agent-Logs-Url: https://github.com/nunit/nunit/sessions/b5ea6ea3-eed1-40bd-924e-f4822f0803fc * Add `mainthread` argument parsing to nunitlite * Update src/NUnitFramework/framework/Attributes/PlatformNames.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix comments * Fix with GetEffectiveProperty * Fix syntax in IncludeExcludeAttribute * Remove unused using * Restore baseclass checking that the constraint expects string arguments. * Clarify intent by replacing anonymous `new` with `new ResultState` * Add SameAs<T> to constraintexpression * Do not unpack an array if it is generic or assignable from IEnumerable<> * Don't unpack array if the target method only expects a single parameter. * Update Cake to version 6.1 (#5201) * Bump cake versions to 6.1 * Update our cakescripts project to match the runtime of the current cake version * Updated minver, made it match with the adapter code, and get rid of the warning. * Fix minver version on std --------- Co-authored-by: Terje Sandstrom <terje@hermit.no> * Optimize the unpack method and add some edge case test coverage for existing behaviour. * Ensure the same exception is thrown today as before * Update partition filter to use our GC polyfill * Update src/NUnitFramework/framework/Internal/Extensions/ArgumentsExtensions.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add test for copilot-identified feedback * Moved benchmark code to NUnit.InternalTools repo, and removed it from… (#5203) * Moved benchmark code to NUnit.InternalTools repo, and removed it from framework * Fixing copilot review comments, removing benchmark related stuff in some other files. * Updated third party dev packages licence and versions * Add test coverage for platform attribute behaviours * Respond to feedback * Use non-standard names to augment PlatformHelper. That way it won't affect the standard operation * Add real os specific tests * Add examples of test with Excludes * Fixes #5211 * Add missing invalid depth guard and test coverage of generic configuration * Changed to more informative exception type * Review changes * Exposed the last Recorded Exception in TestResult (#5217) * Exposed the last Recorded Exception in TestResult * Update src/NUnitFramework/tests/TestContextTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Display values of properties in case of an unexpected Exception (#5216) * Display values of properties in case of an unexpected Exception * Cache Exception properties and don't add overridden base class properties * Phase 1 - moving towards dotnet build (#5210) * Phase 1 - moving towards dotnet build * Update src/NUnitFramework/Directory.Build.props Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removed Version property * fixed unreadable logs (blue color) and added copilot deletions to the target file * updated minver * Changed to correct sdk for building, kept an error to check the output * Updated yml * adding a compiler error * fix compiler error again * Adding caching * remove caching * Updated tests that were flaky --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Phase 2 - Issue5207 simplify testing and building in cake (#5213) * Phase 1 - moving towards dotnet build * Update src/NUnitFramework/Directory.Build.props Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removed Version property * fixed unreadable logs (blue color) and added copilot deletions to the target file * updated minver * Fixing cake builds and converting testing * fixing action workflows * fixed copilot comments * fixed copilot comments * increased time for LargeStringCollectionsInReversedOrder * increased time for LargeStringCollectionsInSameOrder * increased time for LargeStringCollectionsInSameOrder * removed SA1001, it crashes with VS formatting * Increased timeout so that CI builds works, they run fast locally * removed windows runs from linux and mac * removed windows runs from linux and mac * reducing CI noise * reducing CI noise * fix flaky windows test * fix other flaky windows tests * Revert "removed SA1001, it crashes with VS formatting" This reverts commit 6547a48. * Fix so that VS and SA agree on formatting. * Changed to correct sdk for building, kept an error to check the output * Updated yml * adding a compiler error * fix compiler error again * Adding caching * Adding trx reporting * removed cache again * Changed to Dorny due to container usage of enrico * changed to minimal * Changed failed on error * trying to get linux/mac to work too * Fixed error in reporting * Getting back a bit more than quiet * adjusting verbosity * changed output from tests * reverted change to test that didnt actually work, it might still be flaky * Updated Ci workflow * updated workflow * finickling the workflow again * converted to python and fixed paths * adding debugging * converted to python and fixed paths * adding debugging 2 * test became wrong, now fails correctly * fixing linux/mac * fixing stacktrace * fixing header * fixing stacktrace * Set the delibertaelyfailing test back to Explicit * Fix null reference warning in TestResultsParser * Remove netstandard/netcoreapp from FrameworkPattern Drop IgnoreCase and ToLower as frameworks are case-sensitive. We also only use lower case consistently. * Making flaky test explicit * Reverted TextOutputTests * UPdated building instructions * Updated adapter to 6.2 for all but the net6 which uses 5.2 * Tried to make the DelayedConstraintTests a bit less flaky * New attempt to make it work on linux/mac, * And again * Set TestingPlatformDotnetTestSupport=true, and simplified conditionals * and again, MTP is a mess * Reverted back to adapter 5.2.0 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Issue4824 replace delegates (#5196) * Remove Obsolute Apply(ref TActual) * Add overloads for Action and Func<Task> This is meant to replace TestDelegate and AsyncTestDelegate respectively. To prevent overload resolution conflicts with Lambda, the old methods are deprioritized * Add overloads for Func<TActual> * Add NUnit.Analyzer suppressions for Func<T> Revert when nunit/nunit.analyzers#982 is released * Mark Delegate declarations as Obsolete * Remove some test usages of delegates * Fix merge conflicts after rebase * Fix merge conflicts after rebase --------- Co-authored-by: Terje Sandstrom <terje@hermit.no> * Fastpaths for equivalency + super/sub set tests for certain types (#5224) * Add generic collectiontally and supported classes. * Move the tally fastpath into a base class so that sub/super sets can use it too * A bit of cleanup before trying to fix O(n^2) impl * O(n+m) optimization * Fastpath for bytes * Add "unsorted" dictionary-based path. 5x faster for strings * Add copilot instruction to preserve performance * Convert CollectionTally{T} to use strategy pattern internally with different paths for primitives or others * Lazily instantiate the strategies * A bit of usability cleanup for CollectionTally{T} * Make the generic collectiontally{t} work for types not explicitly targeted * Fix a few bugs * Better encapsulate the state management required by the collection tally strategies and ensure all paths go to new collection tally class * Mark the public protected method obsolete * Use AssignableFrom for sortable check since it's available on all runtimes and still intrinsic * Update the collectiontally tests to target the generic version * Lower the largecollection timing thresholds by factor of 5 since the perf improvements seem to speed it up by at least factor of 6 * Re-add stringcollection special case as it previously existed in non-generic CollectionTally * Add test coverage for generic IEnumerable<T>.IsSortable() method and for some additional collectiontally paths * Update src/NUnitFramework/framework/Constraints/CollectionTally{T}.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/NUnitFramework/tests/Constraints/CollectionEquivalentConstraintTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/NUnitFramework/framework/Constraints/CollectionTally{T}.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/NUnitFramework/tests/Internal/Extensions/IEnumerableExtensionTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Improve sort-related fastpath inference not types not explicity targeted * Fix some outstanding bugs when dealing with object-based collections to maximize the chosen strategy * Make some new things internal instead of public. Make it so tuples + valuetuples can use more efficient strategies if their underlying types support it * Allow the NUnit Comparer adapter to adhere to hashcode/equality semantics * Update src/NUnitFramework/framework/Constraints/NUnitEqualityComparer.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Rename HashCodeExtensions.cs to HashCode.cs to match the contained type Agent-Logs-Url: https://github.com/nunit/nunit/sessions/f62a73e3-7ab5-4940-8412-0921b378452c Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> * Remove the need to base a base class field to a base class method. * Fix strangling between CollectionTally and ItemsStrategy This by moving fields from the CollectionTally class to the ItemsStrategy class. * Update src/NUnitFramework/tests/Constraints/CollectionEquivalentConstraintTests.cs Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Revert "Fix strangling between CollectionTally and ItemsStrategy" This reverts commit b9b7e90. * Remove tolerance from the equality adapter for now * Make arguments consistent in test helper * Move instantiation back to a SetUp callback * Remove unneccesary check on _comparePropertiesConfiguration * Add descriptive comment for why to target a few explicit type fast-paths * Re-add descriptive elapsed time for the equivalency performance tests * Rename "MergeSortItemsStrategy" to "SortAndScanItemsStrategy" for better clarity * Remove null default arg. It's unused * Add test for nulls * Explicitly allow for nullable missing/extra items * Avoid accessing private variables from both class and inner class --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> * Reorganize collectionequiavelncy tests so that only performance-focused ones are excluded from MacOS. * Enable collectionequivalency tests on MacOS * Make TestParameters.ArgDisplayNames getter public, #3822 (#5236) * Adjust a few cases for passing exceptions to AfterTest hook * Add coverage for capturing exception for setup/teardown * After exception-related test cases for hooks and test actions * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove unused result variables to fix CS0219 warnings-as-errors build failure Agent-Logs-Url: https://github.com/nunit/nunit/sessions/cf703463-aadb-4e1b-9dfc-e810ffaba573 Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> * Ensure the approach to testing path constraints is consistent across OSes and sub-constraint * chore: Bump NUnit.Analyzers and remove TODOs (#5248) * fix RetryAttribute * add unit test * improve unit test * Apply suggestions from code review Co-authored-by: Terje Sandstrom <terje@hermit.no> * fix `RetryOnAllowedStopsOnOtherException` and add `using (Assert.EnterMultipleScope())` to applicable tests in `RetryAttributeTests` * apply suggestions * Refactor DateTimeOffsetEquality tests to use explicit [Test] methods to avoid invalid cases being generated from data points * Reduce skipped count by filtering out invalid test cases for runtimeframework ParseByClrVersion tests * Remove tests duplicated by DatapointTests.WorksOnField * Move the nullvaluesource tests into testdata so that we can remove the "explicit" usage on the original test method and prevent them reporting as "skipped" * Respond to most feedback * Merged with main, and fixed #5253 * Fixed some comments * Apply suggestion from @stevenaw * Apply suggestions from code review Co-authored-by: Steven Weerdenburg <stevenaw@users.noreply.github.com> * Fix up a few things from the in-browser suggestions. It all passed locally * Update src/NUnitFramework/tests/AsyncExecutionApiAdapter.Constraint-based.cs Co-authored-by: Steven Weerdenburg <stevenaw@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Fixed comments * Fix comment * Fix nullanble issue --------- Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Co-authored-by: stevenaw <sweerdenburg@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Steven Weerdenburg <stevenaw@users.noreply.github.com> Co-authored-by: Will Dean <will.dean@indcomp.co.uk> Co-authored-by: Paul Irwin <paulirwin@gmail.com> Co-authored-by: Mikkel Nylander Bundgaard <mikkelbu@users.noreply.github.com> Co-authored-by: ackhack <christoph.fuerbacher@gmail.com> Co-authored-by: Christoph Fürbacher <43985368+ackhack@users.noreply.github.com> * fixed compiler errors * fixed 4th copilot comment on PR * fixed 3rd copilot comment on PR --------- Co-authored-by: stevenaw <sweerdenburg@gmail.com> Co-authored-by: Mikkel Bundgaard <mikkelbu@gmail.com> Co-authored-by: Jonathan Gilbert <logic@deltaq.org> Co-authored-by: Manfred Brands <manfred-brands@users.noreply.github.com> Co-authored-by: Mikkel Nylander Bundgaard <mikkelbu@users.noreply.github.com> Co-authored-by: Steven Weerdenburg <stevenaw@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stevenaw <439844+stevenaw@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Will Dean <will.dean@indcomp.co.uk> Co-authored-by: Paul Irwin <paulirwin@gmail.com> Co-authored-by: ackhack <christoph.fuerbacher@gmail.com> Co-authored-by: Christoph Fürbacher <43985368+ackhack@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
… framework
Fixes #5198