NETStandard 1.3 support dropped in NUnit 3.10

Hello,
The support for .NET Standard 1.3 seems to be dropped in NUnit 3.10, and now the lowest .NET Standard is 1.6, which does not work with Universal when targeting RS1 or RS2 (the latest version that works is 1.4). I might have missed it, but I did not find any notes on why .NET Standard 1.3 support is dropped. So, just wondering what options there might be to for RS1/RS2? Could we bring back earlier .NET Standard support?

@jrjrguo

Hi @jrjrguo
There was a lot of discussion of this in #2380. Below I've found some of the comments mentioning 1.3 (especially the later ones are relevant). That being said, I'm no expert in .NET Standard, so for answering your question, I think we will have to wait for either @jnm2 or @rprouse.

#2380 (comment)
#2380 (comment)
#2380 (comment)
#2380 (comment)
#2380 (comment)
#2380 (comment)

@jrjrguo

Wow Mikkel! Thanks for the quick links!

@jrjrguo netstandard1.3 presented a significant amount of work and the only platform loss was dropping UWP 10 in favor of UWP 10.0.16299. I believe we'd all like to support any platform on which people are actually running their tests.

What is your test project's target framework, uwp10.0?
If so, then we'd most likely be deciding whether we want to add (and specially test) a uwp10.0 build or a netstandard1.4 build.

@jrjrguo

Part of the decision to drop 1.3 was we only had very rudimentary support for running UWP 10 tests and nobody was using that runner. You had to compile a Xamarin.Forms test app running UWP and manually run the tests.

@jrjrguo are you upgrading an existing project? If so, what were you using to run your UWP tests?

@jnm2

Thanks for all the replies!
@jnm2 Yes, we are targeting uwp10.0 and we need to keep supporting it.
@rprouse We are not using nunit.xamarin to run UWP tests currently. We have our own uwp test runner that is still depending on NUnit 3.6.1, but we do want to switch to nunit.xamarin for all platforms.

@jrjrguo

uap10.0 is still very relevant, because it's the only way to build a universal app that runs on Windows 10 that is older than the Windows 10 Fall Creators Update from October 2017. The thing is that the Windows that is installed on Microsoft Surface Hubs is based on the Windows 10 Creators Update from April 2017, and that is not going to change quickly. So if you want to build Universal Apps in C# that run on Surface Hubs, then you have to use uap10.0 for the foreseeable future.

This is the case for us. I'm working at Microsoft in a team with @jrjrguo, we're around 80 developers and we have to target uap10.0 We're using NUnit to run tests on uap10.0 , using a in-house test runner. We'd love to keep using an up-to-date NUnit (and possibly switch to the NUnit.Xamarin runner).
For that, how much effort would it be possible to bring the netstandard1.4/netstandard1.3 build back?

I'm looking at it now. netstandard1.4 is much less complex to build for than uap10.0, especially considering that we compile cross-platform. Depending what the difference to NUnit is between 1.4 and 1.6, we might not keep both.

@rprouse

The differences between .NET Standard1.4 and 1.6 can be easily bridged with reflection. @rprouse, what do you think? If we do 1.4 in place of 1.6, we won't have to figure out how to do a proper csproj test matrix. 😇

https://github.com/nunit/nunit/compare/jnm2/netstandard1.4

First commit adds netstandard1.4 support and builds successfully.
Second commit removes netstandard1.6 and builds successfully.
Tests don't run yet, was a little surprised to see this:

System.Reflection.TypeExtensions is not supported on NET Standard 1.3 or 1.5.

Looks like it means I'll have to put NUnit.Compatibility back...

@jnm2

Wow, I did not expect such quick actions. Thanks for looking into it @jnm2!

It seems like some sort of shortcut has been taken in System.Reflection.TypeExtensions 4.5. Had a moment of inspiration and tried 4.4 which resulted in all tests running and passing on the first shot. 🎉

#2894

The netstandard1.4 nupkg is building fine in my ad-hoc uap10.0 and netcoreapp1.1 test projects.

It seems like this will be a viable way of running acceptance tests on the nupkg itself as a build step after package: https://github.com/nunit/nunit/tree/jnm2/nupkg_tests/tests

I just need more time to learn how to drive UWP tests from the command line.

It seems like some sort of shortcut has been taken in System.Reflection.TypeExtensions 4.5. Had a moment of inspiration and tried 4.4 which resulted in all tests running and passing on the first shot. 🎉

Are you referring to the _._ files in the NuGet package here?

I had lots of fun with these in #2237. My understanding is that that file is put in place when the platform doesn't support the .NET Standard API yet - so that the package fails to restore for the platform at restore time, as opposed to throwing errors at runtime.

If that's correct, and the _._ file has been added between 4.4 and 4.5, then I'm not sure reverting to the older package version is necessarily the best solution. It looks like the placeholder was explicitly added in here - but without much background info...maybe some of the UWP bods here may know more?

To clarify - my concern is that depending on v4.4 of the package may allow us to build, but instead expose us to runtime errors. If that's the case - then adding back our own Reflection functionality seems preferable. If I've misunderstood how all this works, then taking on the package dependency seems ideal. 😄

@ChrisMaddock

@ChrisMaddock This scenario seems different. With System.Reflection.TypeExtensions 4.5.0, everything restores and builds without error or warning. It's compiling against real DLLs. A quick look inside just now shows that there are no _._ files. It's only at runtime that it fails.

I just opened https://github.com/dotnet/corefx/issues/30506 to learn more, but in the meantime, the real way to be sure of ourselves will be doing a full restore/build/run of https://github.com/nunit/nunit/tree/jnm2/nupkg_tests/tests after our package build step.

Do we think it's safe to merge #2894 which gets netstandard1.4-compatible packages in our prerelease feed, and add UWP acceptance tests in a later PR?

@jnm2

@jnm2 - Sorry, I pulled the package and saw the placeholder for uap10.0.16299 - but that's the version above the one we're looking at here...isn't it. So that version of UWP would get one of our other .NET Standard builds. My bad, I misunderstood!

Personally - I thought @steffenloesch made a reasonable argument for putting the 1.4 build back - although, Rob, you're correct out UWP audience is small. If we hadn't already done the work, I'm not sure I'd support adding it, but given we only need to add it back - I think it might be worth doing for a little longer. Again - it is a very small audience, however.

@ChrisMaddock

@ChrisMaddock Right. uap10.0.16299 would pick up our netstandard2.0 build, though even if it picked up our netstandard1.4 build, I think we'd be fine because (I think) the same System.Reflection.TypeExtensions.dll is already in-box on uap10.0.16299 and would load just fine. Worth checking out though.

Edit, there's this: https://github.com/NuGet/Home/issues/6201

@steffenloesch

Looking through your PR, I find that I actually like the changes. Shall we go for it? How can we test/verify other than the unit tests? @steffenloesch are you able to verify using our dev build and your custom test runner?

Sweet, sounds like the three of us are in favor. I'll merge and continue working to see how far I get on a UWP acceptance test.

Also, according to https://github.com/dotnet/corefx/issues/30506 there was a mixup in the packaging of System.Reflection.TypeExtensions 4.5.0 and it was supposed to have worked all along.

@rprouse

@rprouse yes we can check if these changes work with our test runner and test suite. We will ping this thread once we have results.

@rprouse

@rprouse @jnm2 We did a quick check and the changes work fine with our test runner!
We did have to make a small modification in order to workaround a runtime issue, but we have seen that in 3.9.0 as well. I will post a separate issue for it.

@ChrisMaddock

Fyi @ChrisMaddock and @rprouse, the UWP acceptance test is shaping up:

nunit/build.cake

Lines 252 to 282 in 40c4db6

    
           Task("Acceptance-uap10.0") 
        
               .WithCriteria(IsRunningOnWindows()) 
        
               .Does(() => 
        
               { 
        
                   DeleteDirectories(new[] 
        
                   { 
        
                       @"tests\Isolated package cache\nunit", 
        
                       @"tests\Isolated package cache\nunitlite" 
        
                   }, new DeleteDirectorySettings { Recursive = true }); 
        
                   MSBuild(@"tests\uap10.0", new MSBuildSettings 
        
                   { 
        
                       WorkingDirectory = @"tests\uap10.0", 
        
                       Verbosity = Verbosity.Minimal, 
        
                       Configuration = "Release", 
        
                       PlatformTarget = PlatformTarget.x86, 
        
                       Restore = true 
        
                   }); 
        
                   MSBuild(@"src\AppxRunner", new MSBuildSettings 
        
                   { 
        
                       Verbosity = Verbosity.Minimal, 
        
                       Configuration = configuration, 
        
                       Restore = true 
        
                   }); 
        
                   StartProcess($@"src\AppxRunner\AppxRunner\bin\{configuration}\net46\AppxRunner.exe", new ProcessArgumentBuilder() 
        
                       .AppendQuoted(@"tests\uap10.0\uap10.0\bin\x86\Release\uap10.0.build.appxrecipe") 
        
                       .Append("NUnit.AcceptanceTests.Universal.App") 
        
                       .Render()); 
        
               });

It's building a UWP app, restoring the latest NUnit and NUnitLite packages from the /package folder while using an isolated cache (for repeatability and to keep your machine clean), building a UWP launcher tool and successfully using it to launch and wait for NUnitLite inside the UWP app.

More soon.

Update on UWP acceptance tests here: #2903

@jnm2

@jnm2 looking good. FYI, I am bike packing until late next week, so feel free to move this forward ok on your own and merge when ready.

Bike packing, sounds like fun!
I have some questions for y'all in that issue but there is no rush. It's not like I can run out of work 😊

Navigation Menu

Repository navigation

NETStandard 1.3 support dropped in NUnit 3.10 #2887

3 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	Task("Acceptance-uap10.0")
	.WithCriteria(IsRunningOnWindows())
	.Does(() =>
	{
	DeleteDirectories(new[]
	{
	@"tests\Isolated package cache\nunit",
	@"tests\Isolated package cache\nunitlite"
	}, new DeleteDirectorySettings { Recursive = true });

	MSBuild(@"tests\uap10.0", new MSBuildSettings
	{
	WorkingDirectory = @"tests\uap10.0",
	Verbosity = Verbosity.Minimal,
	Configuration = "Release",
	PlatformTarget = PlatformTarget.x86,
	Restore = true
	});

	MSBuild(@"src\AppxRunner", new MSBuildSettings
	{
	Verbosity = Verbosity.Minimal,
	Configuration = configuration,
	Restore = true
	});

	StartProcess($@"src\AppxRunner\AppxRunner\bin\{configuration}\net46\AppxRunner.exe", new ProcessArgumentBuilder()
	.AppendQuoted(@"tests\uap10.0\uap10.0\bin\x86\Release\uap10.0.build.appxrecipe")
	.Append("NUnit.AcceptanceTests.Universal.App")
	.Render());
	});

NETStandard 1.3 support dropped in NUnit 3.10 #2887

Description

Activity

mikkelbu commented on Jun 12, 2018

jnm2 commented on Jun 12, 2018

rprouse commented on Jun 12, 2018

jrjrguo commented on Jun 13, 2018

steffenloesch commented on Jun 15, 2018

jnm2 commented on Jun 15, 2018

jnm2 commented on Jun 15, 2018

jnm2 commented on Jun 15, 2018

steffenloesch commented on Jun 15, 2018

jnm2 commented on Jun 19, 2018

jnm2 commented on Jun 19, 2018

ChrisMaddock commented on Jun 19, 2018

jnm2 commented on Jun 19, 2018

3 remaining items

ChrisMaddock commented on Jun 19, 2018

jnm2 commented on Jun 19, 2018

rprouse commented on Jun 19, 2018

jnm2 commented on Jun 20, 2018

jnm2 commented on Jun 20, 2018

steffenloesch commented on Jun 20, 2018

jrjrguo commented on Jun 20, 2018

jnm2 commented on Jun 22, 2018

jnm2 commented on Jun 23, 2018

rprouse commented on Jun 23, 2018

jnm2 commented on Jun 23, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions