Skip to content

Make addTearDown and testWidgets interoperate properly, for invariants checked by testWidgets #123189

@gnprice

Description

@gnprice

(This problem statement is copied from @fzyzcjy's PR #114468, with minor edits.)

Consider this simple example

import 'package:flutter/scheduler.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('addTearDown should work', (tester) async { // <-- THIS FAILS
    timeDilation = 2;
    addTearDown(() => timeDilation = 1);
  });

  testWidgets('directly reset should work', (tester) async { // <-- this is ok
    timeDilation = 2;
    timeDilation = 1;
  });
}

It yields:

Details
/Volumes/MyExternal/ExternalRefCode/flutter/bin/flutter --no-color test --machine --start-paused test/a.dart
Testing started at 09:50 ...

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
The timeDilation was changed and not reset by the test.

When the exception was thrown, this was the stack:
#0      SchedulerBinding.debugAssertNoTimeDilation.<anonymous closure> (package:flutter/src/scheduler/binding.dart:662:9)
#1      SchedulerBinding.debugAssertNoTimeDilation (package:flutter/src/scheduler/binding.dart:665:6)
#2      TestWidgetsFlutterBinding._verifyInvariants (package:flutter_test/src/binding.dart:968:12)
#3      AutomatedTestWidgetsFlutterBinding._verifyInvariants (package:flutter_test/src/binding.dart:1433:11)
#4      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:952:7)
<asynchronous suspension>

The test description was:
  addTearDown should work
════════════════════════════════════════════════════════════════════════════════════════════════════

Test failed. See exception logs above.
The test description was: addTearDown should work

In other words, we do not allow resetting configurations in addTearDown (or tearDown) for anything that testWidgets checks as an invariant at the end of the test. Instead, we must do it at the end of the test body.

IMHO resetting things in addTearDown/tearDown is a commonly seen practice. For example, window.physicalSize can be reset in a tear-down function, and even Flutter test code inside the framework does so a lot of times. A quick search also shows that, such as this example, resetting in tear down holds even for other tests such as Python.

By disallowing so, Flutter devs may have a bit more friction when learning Flutter, since they may firstly write down code that follows common practice, realizing it does not work, and change it.

It is also inconsistent with other parts of the Flutter. As mentioned above, window.physicalSize can be reset in a tear-down function, but things like timeDilation cannot.

Fixing this can also make code a bit simpler. Currently, whenever writing setup code (e.g. timeDilation=2), we must put tear down code at the end of the function, and wrap with a try-finally. But if we fix this issue, the teardown code can be put near the setup code, so it is a bit cleaner for code readers. By the way, this is also a bit like the "defer" keyword in go - something like configure(); defer reset(); other_functions() will let the reset be executed last.

In some cases, this seems to simplify code a lot. For example, in #113830 (comment), I (@fzyzcjy) have to introduce a weird timeDilation reset that seems to have no pairing timeDilation modification (and cause confusion of readers - even code reviewers). If this issue is fixed, the reset can be put to the next line of modification, so it is clear.

(end quote)


The PR #114468 was aimed at fixing this issue. It was merged, but then reverted (#120739).

Issue #121917 is a proposal that would also fix this issue. Quoting @pdblasi-google from #114468 (comment) :

The "Modifications" section of that issue should enable setup and addTearDown to work appropriately. The root problem here is that testWidgets is doing a lot of not-testing in its test call, which leads to things not being set up early enough to use the test library's setup.

When just the actual test is run within the test call, this problem should go away.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work lista: tests"flutter test", flutter_test, or one of our testsframeworkflutter/packages/flutter repository. See also f: labels.team-frameworkOwned by Framework teamtriaged-frameworkTriaged by Framework team

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions