Initial set-up

Check that the IntelliJ Idea UTBot plugin can be successfully installed

• Choose appropriate workflow from the list - with Security Manager included - plugin build used
• Download plugin
• Open IntelliJ IDE
• Remove previously installed UTBot plugin
• Install the downloaded plugin

Go through manual scenarios

Basic security

• Project with JDK 8 + manual scenario from PR Add SecurityManager support to block suspicious code #622 #625 + default settings
• Fuzzing only
• Symbolic execution only
• Project with JDK 11 + default settings, bug found: Socket connect should be sandboxed #792
• Fuzzing only
• Symbolic execution only
• Concrete executor is invoked from Symbolic execution

Project with JDK 17 - blocked as UTBot is incompatible with JDK 12+

Permissions

• add PropertyPermission in ~.utbot\sandbox.policy - check test for property is not in sandbox, execute
• add "permission java.io.FilePermission "<>", "read";" - check test for read not in sandbox, execute
• add AllPermission in ~.utbot\sandbox.policy - check NO tests are in sandbox, try to execute (carefully!!!)
• corrupt permission - check how UTBot fails - found Too many permissions file rereads - can slow down much #796
• empty file
• remove ~.utbot\sandbox.policy file - check generated tests are in sandbox

Test rendering

• JUnit4
• JUnit5
• TestNG
• JUnit5 with parametrization - JUnit5 params dependency is required in default IDEA's Gradle project #620 reproducing
• TestNG with parametrization

Code structures

• in private method
• in static block of a used class - bug found: Implicit unpermitted operations should be sandboxed #791
• in compiler of a used class - bug found: Implicit unpermitted operations should be sandboxed #791
• in a separate thread - not yet supported - enhancement: Disable tests with unpermitted operations in Threads #895