[Swift in WebKit] Integrate Swift Testing into run-api-tests (part 2)#62305
Conversation
|
EWS run on previous version of this PR (hash 6f3fb5a) Details
|
6f3fb5a to
8a7441c
Compare
|
EWS run on previous version of this PR (hash 8a7441c) Details
|
8a7441c to
9a47e11
Compare
|
EWS run on previous version of this PR (hash 9a47e11) Details
|
9a47e11 to
fa248c0
Compare
|
EWS run on previous version of this PR (hash fa248c0) Details
|
fa248c0 to
37e3fd6
Compare
|
EWS run on previous version of this PR (hash 37e3fd6) Details |
emw-apple
left a comment
There was a problem hiding this comment.
Really cool! My inline comments are mostly nits and clarification questions, but I do have a big one to ask up front:
I understand that we need to load the swift-testing runtime to run the tests, and that the patch does this by putting $(DEVELOPER_DIR) into the list of @rpath search paths which are baked into the binary.
Problem is, DEVELOPER_DIR is the path to Xcode on the builder. That path can be relocated, so it seems like this approach would break if a builder and tester have Xcode installed at different paths.
I think webkit.org infra is fine (all the machines we provision use /Applications/Xcode.app), but Apple internal workflows would be affected. It would mean, for instance, that an engineer can't download a prebuilt TestWebKitAPI and run the tests locally.
| extension SwiftTestingABI.__CommandLineArguments_v0: Codable { | ||
| } | ||
|
|
There was a problem hiding this comment.
nit: Do we prefer this instead of just writing : Codable as part of the struct's declaration?
There was a problem hiding this comment.
We do; in general, Swift convention prefers breaking out semantically different parts of logic into extensions, especially when those involve conformances to protocols
There was a problem hiding this comment.
I guess I just don't see the semantic difference that would separate breaking things apart. These are all data types, so implementing Codable seems like a core part of the type's behavior. Doesn't matter very much to me though
37e3fd6 to
bbe2c14
Compare
|
EWS run on previous version of this PR (hash bbe2c14) Details
|
bbe2c14 to
12f95cd
Compare
|
EWS run on previous version of this PR (hash 12f95cd) Details
|
bdd99ed to
e6863be
Compare
|
EWS run on previous version of this PR (hash e6863be) Details |
e6863be to
29eb517
Compare
|
EWS run on previous version of this PR (hash 29eb517) Details
|
29eb517 to
209277d
Compare
|
EWS run on previous version of this PR (hash 209277d) Details
|
209277d to
a8ca5f3
Compare
|
EWS run on previous version of this PR (hash a8ca5f3) Details
|
a8ca5f3 to
7900b98
Compare
|
EWS run on previous version of this PR (hash 7900b98) Details
|
7900b98 to
8785f36
Compare
|
EWS run on previous version of this PR (hash 8785f36) Details
|
8785f36 to
000d747
Compare
|
EWS run on previous version of this PR (hash 000d747) Details
|
000d747 to
92b7460
Compare
|
EWS run on previous version of this PR (hash 92b7460) Details
|
92b7460 to
df9d137
Compare
|
EWS run on current version of this PR (hash df9d137) Details
|
| extension SwiftTestingABI.__CommandLineArguments_v0: Codable { | ||
| } | ||
|
|
There was a problem hiding this comment.
I guess I just don't see the semantic difference that would separate breaking things apart. These are all data types, so implementing Codable seems like a core part of the type's behavior. Doesn't matter very much to me though
https://bugs.webkit.org/show_bug.cgi?id=311769 rdar://174358586 Reviewed by Elliott Williams. Adds the ability for `run-api-tests` to also include and run tests written with Swift Testing. For now, this PR limits the scope to just those tests in `WKWebViewSwiftOverlayTests.swift` for validation purposes; a subsequent PR will enable this for all the existing Swift Testing tests. * Tools/Scripts/webkitpy/api_tests/manager.py: (Manager._test_list_from_output): (Manager._find_test_subset): (Manager._collect_tests): * Tools/Scripts/webkitpy/api_tests/runner.py: (_Worker._run_single_test): Modify the `run-api-tests` script to handle Swift Testing tests; specifically their different naming scheme compared to gtest tests. * Tools/Scripts/webkitpy/api_tests/manager_unittest.py: (test_test_list_from_output_mixed_gtest_and_swift): (test_find_test_subset_gtest_full_name): (test_find_test_subset_gtest_suite_and_test): (test_find_test_subset_gtest_binary_and_suite): (test_find_test_subset_gtest_suite_only): (test_find_test_subset_gtest_wildcard): (test_find_test_subset_swift_full_name): (test_find_test_subset_swift_suite_and_test): (test_find_test_subset_swift_binary_and_suite): (test_find_test_subset_swift_suite_only): (test_find_test_subset_swift_wildcard): (test_find_test_subset_parentheses_are_literal): (test_find_test_subset_no_match): Add new tests for the script. * Tools/TestWebKitAPI/Configurations/TestIPC.xcconfig: * Tools/TestWebKitAPI/Configurations/TestWGSL.xcconfig: * Tools/TestWebKitAPI/Configurations/TestWTF.xcconfig: * Tools/TestWebKitAPI/Configurations/TestWebKitAPI.xcconfig: * Tools/TestWebKitAPI/Configurations/TestWebKitAPIBase.xcconfig: Link Testing and update the LD_RUNPATH_SEARCH_PATHS as needed. * Tools/TestWebKitAPI/Runner/GoogleTestsController.swift: Drive-by cleanup: Make a function `private` * Tools/TestWebKitAPI/Runner/SwiftTestingABI.swift: Added. (__CommandLineArguments_v0.verbosity): (__CommandLineArguments_v0.listTests): (__CommandLineArguments_v0.filter): (__CommandLineArguments_v0.skip): (__CommandLineArguments_v0.repetitions): (__CommandLineArguments_v0.parallel): (ID.stringValue): (ID.encode(to:)): * Tools/TestWebKitAPI/Runner/SwiftTestsController.swift: Added. (Storage.tests): (Storage.events): (saveRecord(_:)): (writeOutputIfNeeded(_:)): (handleRecord(_:)): (run(with:)): (SwiftTestingABI.canonicalizedRepresentation): * Tools/TestWebKitAPI/Runner/TestWebKitAPI.swift: (TestWebKitAPI.run): * Tools/TestWebKitAPI/Runner/TestWebKitAPISupport.h: * Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: Add a new `TestRunner` type that is responsible for running Swift Testing types from within the binary directly. This uses the Swift Testing ABI types and functions. Canonical link: https://commits.webkit.org/311244@main
df9d137 to
c5980c2
Compare
|
Committed 311244@main (c5980c2): https://commits.webkit.org/311244@main Reviewed commits have been landed. Closing PR #62305 and removing active labels. |
🛠 ios-apple
c5980c2
df9d137
🛠 win🧪 win-tests🧪 ios-wk2-wpt🧪 api-mac-debug🧪 api-ios🧪 mac-AS-debug-wk2