Fixes #2648 : Add support for customising strictness via @Mock annotation and MockSettings by sivaprasadreddy · Pull Request #2650 · mockito/mockito

sivaprasadreddy · 2022-05-23T04:18:41Z

This PR fixes the issue #2648.

This is to add support for customising strictness level for mock using @mock annotation or MockSettings.

Example 1:

class MyTest {
     @Mock(strictness = Strictness.LENIENT)
     Foo foo;
     ....
}

Example 2:

class MyTest {
     Foo mock = Mockito.mock(Foo.class, withSettings().strictness(Strictness.WARN));
     ....
}

…tion and MockSettings

…epresentation

codecov-commenter · 2022-05-26T09:05:25Z

Codecov Report

Attention: Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.

Project coverage is 86.22%. Comparing base (d7a8ae0) to head (d090780).
Report is 458 commits behind head on main.

Files with missing lines	Patch %	Lines
.../mockito/internal/stubbing/StrictnessSelector.java	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2650      +/-   ##
============================================
- Coverage     86.24%   86.22%   -0.03%     
- Complexity     2761     2763       +2     
============================================
  Files           314      314              
  Lines          8282     8290       +8     
  Branches       1031     1031              
============================================
+ Hits           7143     7148       +5     
- Misses          872      873       +1     
- Partials        267      269       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

TimvdLippe

Nice work! It's already looking good, mostly some nits. After that, this is ready to merge 🎉

TimvdLippe · 2022-05-26T13:08:21Z

+     * Mock will have custom strictness, see {@link MockSettings#strictness(Strictness)}.
+     * For examples how to use 'Mock' annotation and parameters see {@link Mock}.
+     *
+     * @since 4.5.2


Nit: @since 4.6.0 since we will include this in the next minor release

TimvdLippe · 2022-05-26T13:09:50Z

+
+    /**
+     * Specifies strictness level for the mock.
+     * The default strictness level is STRICT_STUBS


Nit: the default is the default of the runner. If you use the lenient runner, it will be lenient. I suggest to formulate it the following way:

The default strictness level is determined by the rule/runner used. If you are using no rule/runner, the default strictness level is LENIENT.

TimvdLippe · 2022-05-26T13:11:33Z

+     * Sets strictness level for the mock, e.g. having {@link Strictness#STRICT_STUBS} characteristic.
+     * For more information about using mocks with custom strictness, see {@link MockSettings#strictness(Strictness)}.
+     *
+     * @since 4.5.2


TimvdLippe · 2022-05-26T13:15:37Z

+ * <h3 id="52">52. <a class="meaningful_link" href="#mockito_strictness" name="mockito_strictness">
+ *  New strictness attribute for @Mock annotation and <code>MockSettings.strictness()</code> methods (Since 4.5.2)</a></h3>
+ *
+ * By default Mocks use Strict stubbing and now you can customize the strictness level


Nit: mocks by default are lenient. I would omit that part altogether and reword it to:

You can now customize the strictness level for a single mock, either using `@Mock` annotation strictness attribute or using `MockSettings.strictness()`. This can be useful if you want all of your mocks to be strict, but one of the mocks to be lenient.

TimvdLippe · 2022-05-26T13:17:18Z

+
+        // but regular mock throws:
+        Assertions.assertThatThrownBy(
+                        new ThrowableAssert.ThrowingCallable() {


Nit: I think we can use a lambda here, can we not?

() -> ProductionCode.simpleMethod(regularMock, "4")

TimvdLippe · 2022-05-26T13:17:30Z

+        when(regularMock.simpleMethod("3")).thenReturn("3");
+        when(strictMock.simpleMethod("2")).thenReturn("2");
+
+        // then lenient mock does not throw:


Same here, let's have 3 separate 3 test cases here.

Separated this into multiple tests.

TimvdLippe

Thanks for the PR and great implementation 🎉

chadlwilson · 2022-05-30T06:06:00Z

+    }
+
+    @Test
+    public void mock_is_strict_with_default_settings() {


Quick Q - was it intentional for this to change mocks defined by mock(Blah.class) i.e with no rule/Junit MockitoSettings to strict by default?

I believe the previous behaviour was lenient by default?

We have not changed the behavior here. This test is using a Strict rule explicitly (https://github.com/mockito/mockito/pull/2650/files/d0907808e9fce24f6dcddc94d1ea2c3ebfa15676#diff-3040268f6b18ef562dd495ac7fc13465723311889d5f326bfb615cc3aed37415R21) to verify that we handle that case correctly.

In case you are hitting any regressions, please file a new issue. You are most likely running into a different problem.

OK, thanks. We definitely have regressions (previously lenient mocks now behaving as strict) but I may have misidentified the possible cause. Will need to dig further if I get time.

Another person has beat me to raising it at #2656 but this seems to be the same issue we are having.

@mock

## Changes - **mockito: 4.4.0 -> 4.6.1** (https://github.com/mockito/mockito/releases) Most important updates: - Fixes mockito/mockito#2648 : Add support for customising strictness via @mock annotation and MockSettings mockito/mockito#2650 ## Why is this change needed? According to the [Mockito documentation](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#when(T)) : > Although it is possible to verify a stubbed invocation, usually it's just redundant. Let's say you've stubbed foo.bar(). If your code cares what foo.bar() returns then something else breaks(often before even verify() gets executed). If your code doesn't care what get(0) returns then it should not be stubbed. While working on the [Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest ](https://issues.apache.org/jira/browse/KAFKA-12947) I noticed that described behavior wasn't applied when you create a new `mock` like this. ```java final Metrics metrics = mock(Metrics.class); when(metrics.metric(metricName)).thenReturn(null); ... invoke SUT verify(metrics).metric(metricName); // this should be redundant (according to docs) ``` After further investigation I figured out that described behaviour wasn't implemented until`v4.6.1`. With this change we are now able to mock objects like this: ```java Foo explicitStrictMock = mock(Foo.class, withSettings().strictness(Strictness.STRICT_STUBS)); ``` - link to docs: [MockSettings.html#strictness](https://javadoc.io/static/org.mockito/mockito-core/4.6.1/org/mockito/quality/Strictness.html#STRICT_STUBS) It looks like I can accomplish the same thing by using the `@RunWith(MockitoJUnitRunner.StrictStubs.class) ` instead of the `@RunWith(MockitoJUnitRunner.class)` so mockito dependency version update is not mandatory, but it would be nice to stay up-to-date and use the latest version (it's up to MR reviewer to decide if we are going to merge this now, or just close the MR and update mockito version later). Reviewers: Ismael Juma <ismael@juma.me.uk>

@mock

## Changes - **mockito: 4.4.0 -> 4.6.1** (https://github.com/mockito/mockito/releases) Most important updates: - Fixes mockito/mockito#2648 : Add support for customising strictness via @mock annotation and MockSettings mockito/mockito#2650 ## Why is this change needed? According to the [Mockito documentation](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#when(T)) : > Although it is possible to verify a stubbed invocation, usually it's just redundant. Let's say you've stubbed foo.bar(). If your code cares what foo.bar() returns then something else breaks(often before even verify() gets executed). If your code doesn't care what get(0) returns then it should not be stubbed. While working on the [Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest ](https://issues.apache.org/jira/browse/KAFKA-12947) I noticed that described behavior wasn't applied when you create a new `mock` like this. ```java final Metrics metrics = mock(Metrics.class); when(metrics.metric(metricName)).thenReturn(null); ... invoke SUT verify(metrics).metric(metricName); // this should be redundant (according to docs) ``` After further investigation I figured out that described behaviour wasn't implemented until`v4.6.1`. With this change we are now able to mock objects like this: ```java Foo explicitStrictMock = mock(Foo.class, withSettings().strictness(Strictness.STRICT_STUBS)); ``` - link to docs: [MockSettings.html#strictness](https://javadoc.io/static/org.mockito/mockito-core/4.6.1/org/mockito/quality/Strictness.html#STRICT_STUBS) It looks like I can accomplish the same thing by using the `@RunWith(MockitoJUnitRunner.StrictStubs.class) ` instead of the `@RunWith(MockitoJUnitRunner.class)` so mockito dependency version update is not mandatory, but it would be nice to stay up-to-date and use the latest version (it's up to MR reviewer to decide if we are going to merge this now, or just close the MR and update mockito version later). Reviewers: Ismael Juma <ismael@juma.me.uk>

Fixes #2648 : Add support for customizing strictness via @mock annota…

2e0390a

…tion and MockSettings

sivaprasadreddy mentioned this pull request May 24, 2022

Any way to enable Strict Stubbing when using Mockito.mock() without using @Mock? #2648

Closed

sivaprasadreddy added 2 commits May 26, 2022 05:53

Fixes #2648 : Update Javadoc comment to escape @ symbol

5835229

Fixes #2648 : Update Javadoc comment to escape @ symbol using octal r…

ff8a4ee

…epresentation

TimvdLippe requested changes May 26, 2022

View reviewed changes

sivaprasadreddy added 2 commits May 26, 2022 21:37

Fixes #2648 : Updated addressing review comments.

1ffc0f0

Fixes #2648 : Applying google java formatting.

d090780

TimvdLippe approved these changes May 26, 2022

View reviewed changes

TimvdLippe merged commit 70cf2d2 into mockito:main May 26, 2022

TimvdLippe mentioned this pull request May 26, 2022

Added support for updating releases shipkit/shipkit-changelog#124

Merged

chadlwilson reviewed May 30, 2022

View reviewed changes

chadlwilson mentioned this pull request May 30, 2022

Bump mockito-core from 4.5.1 to 4.6.0 gocd/gocd#10479

Closed

picimako mentioned this pull request May 30, 2022

Support strictness for mock creation settings picimako/mockitools#24

Closed

This was referenced May 31, 2022

Regression? Strictness set in @MockitoSettings ignored after upgrade from 4.5.1 to 4.6.0 #2656

Closed

Fix for Issue2656 #2659

Merged

dplavcic mentioned this pull request Jul 29, 2022

MINOR: Upgrade mockito test dependencies apache/kafka#12460

Merged

3 tasks

Uh oh!

Conversation

sivaprasadreddy commented May 23, 2022

Uh oh!

codecov-commenter commented May 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

TimvdLippe left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TimvdLippe left a comment

Choose a reason for hiding this comment

Uh oh!

chadlwilson May 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov-commenter commented May 26, 2022 •

edited

Loading

chadlwilson May 30, 2022 •

edited

Loading