Updated windows advanced addon to replace ai Request with ocr endpoints.#364
Updated windows advanced addon to replace ai Request with ocr endpoints.#364ManojTestsigma wants to merge 2 commits intodevfrom
Conversation
📝 WalkthroughWalkthroughThis pull request significantly extends the Windows Advanced and Windows Lite addons by reorganizing utility classes, introducing OCR-based text detection, and adding dozens of new action implementations for keyboard input, image/text clicking, data verification, and file operations. The artifact version is downgraded from 1.0.23 to 1.0.1. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 8
Note
Due to the large number of review comments, Critical severity comments were prioritized as inline comments.
🟠 Major comments (28)
windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/AddDataToFile.java-43-55 (1)
43-55:⚠️ Potential issue | 🟠 MajorNull check occurs after potential NPE.
If
getValue()returns null, calling.toString()on lines 43-44 will throwNullPointerExceptionbefore the null check on line 50 is reached. BothfileContentsandfilePathvalues should be validated for null before conversion.🐛 Proposed fix with proper null validation
try { - String content = fileContents.getValue().toString(); - String path = filePath.getValue().toString(); - - logger.info("Adding data to file: " + path); - logger.info("Data to add: " + content); - - // Validate file path - if (path == null || path.trim().isEmpty()) { + Object contentValue = fileContents.getValue(); + Object pathValue = filePath.getValue(); + + // Validate inputs before conversion + if (pathValue == null || pathValue.toString().trim().isEmpty()) { setErrorMessage("File path cannot be empty"); ScreenshotUtils.captureAndUploadScreenshot(testStepResult, "add_data_to_file_failure_screenshot", logger); return Result.FAILED; } + + String path = pathValue.toString(); + String content = contentValue != null ? contentValue.toString() : ""; + + logger.info("Adding data to file: " + path); + logger.info("Data to add: " + content);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/AddDataToFile.java` around lines 43 - 55, The code calls fileContents.getValue().toString() and filePath.getValue().toString() before validating for null; update AddDataToFile so you first validate fileContents and filePath and their getValue() results for null/empty and only then call toString() on them. Specifically, check fileContents == null or fileContents.getValue() == null (and similarly for filePath) and on failure call setErrorMessage(...) and ScreenshotUtils.captureAndUploadScreenshot(testStepResult, "add_data_to_file_failure_screenshot", logger) and return Result.FAILED; only after these checks convert values to String and proceed, keeping logger.info(...) and other logic unchanged.windows_advanced_actions/src/main/java/com/testsigma/addons/util/StringCompareUtil.java-10-54 (1)
10-54:⚠️ Potential issue | 🟠 MajorNull pointer exception risk if either input string is null.
If
string1orstring2isnull, calling.equals(),.equalsIgnoreCase(),.contains(), or.toLowerCase()will throw aNullPointerException. Add null checks at the start of the method.🛡️ Proposed fix to add null validation
public boolean performOperation(String string1, String string2, String operation){ + if (string1 == null || string2 == null) { + this.errorMessage = "Input strings cannot be null. string1: " + string1 + ", string2: " + string2; + return false; + } boolean equalsCheck = false; boolean containsCheck = false;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/util/StringCompareUtil.java` around lines 10 - 54, performOperation risks NullPointerException when string1 or string2 is null; add early null validation in performOperation to check both inputs before the switch, set this.errorMessage (e.g., "Input string1 is null" or "Input string2 is null" or a combined message) and return false if either is null so no .equals/.contains/.toLowerCase calls run on null; update handling for all branches (equals, equals ignore-case, contains, contains ignore-case) to rely on the validated non-null inputs and preserve setting this.successMessage/this.errorMessage and the boolean result logic.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ExtractDataFromScreen.java-125-141 (1)
125-141:⚠️ Potential issue | 🟠 MajorPotential
NullPointerExceptionwhen JSON fields are missing.
JsonNode.get()returnsnullwhen the field doesn't exist. CallingasBoolean()orasText()on anullreference will throwNullPointerException. Usepath()instead, which returns aMissingNodethat safely handles default values.🐛 Proposed fix using path()
private AIResponse parseAIResponse(String aiResponse) { try { ObjectMapper mapper = new ObjectMapper(); JsonNode jsonNode = mapper.readTree(aiResponse); AIResponse response = new AIResponse(); - response.isQueryRelated = jsonNode.get("isQueryRelated").asBoolean(); - response.extractedText = jsonNode.get("extracted Text").asText(""); - response.additionalData = jsonNode.get("additional data").asText(""); + response.isQueryRelated = jsonNode.path("isQueryRelated").asBoolean(false); + response.extractedText = jsonNode.path("extracted Text").asText(""); + response.additionalData = jsonNode.path("additional data").asText(""); return response; } catch (Exception e) { logger.warn("Failed to parse AI response JSON: " + e.getMessage()); logger.warn("Raw AI response: " + aiResponse); return null; } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ExtractDataFromScreen.java` around lines 125 - 141, The parseAIResponse method can NPE when fields are absent because jsonNode.get(...) may return null; change jsonNode.get(...) calls to jsonNode.path(...) in parseAIResponse so MissingNode handles defaults (e.g., response.isQueryRelated = jsonNode.path("isQueryRelated").asBoolean(); response.extractedText = jsonNode.path("extracted Text").asText(""); response.additionalData = jsonNode.path("additional data").asText("");), keep the same AIResponse field assignments and existing exception handling/logging with logger unchanged.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnImage.java-58-61 (1)
58-61:⚠️ Potential issue | 🟠 MajorDon’t log the screenshot upload URL.
testStepResult.getScreenshotUrl()is typically a presigned upload URL. Writing it at info level exposes temporary upload credentials in the step logs.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnImage.java` around lines 58 - 61, The code logs a presigned upload URL (testStepResult.getScreenshotUrl()) which exposes temporary credentials; in ClickOnImage (where logger.info is called around testStepResult.getScreenshotUrl() and before ocr.uploadFile) remove or redact those logger.info calls so the full URL is not written to logs (either delete the two logger.info lines that reference getScreenshotUrl() or replace them with a non-sensitive message like "Uploading screenshot to storage" without including the URL); ensure no other logging in the ClickOnImage flow outputs testStepResult.getScreenshotUrl().windows_advanced_actions/src/main/java/com/testsigma/addons/util/KeyboardUtils.java-60-100 (1)
60-100:⚠️ Potential issue | 🟠 MajorHandle unmappable characters and supplementary code points before calling
Robot.This method accepts characters via
Character.isLetter()that cannot be typed on any keyboard layout. Characters like é, à, ñ, and CJK ideographs passisLetter()butKeyEvent.getExtendedKeyCodeForChar()returnsVK_UNDEFINEDwhen they lack a key mapping, leading to undefined Robot behavior. Additionally, supplementary Unicode code points (U+10000 and above) are represented as surrogate pairs in Java'schartype; passing them one character at a time causes each surrogate to fail validation and throwIllegalArgumentExceptionmid-input, leaving Robot in an inconsistent state.Add early validation to reject unmappable characters or accept only BMP code points. For full Unicode support, accept String input and handle surrogate pairs correctly using
Character.toCodePoint()andCharacter.codePointAt().🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/util/KeyboardUtils.java` around lines 60 - 100, typeCharacter(Robot robot, char character) currently assumes every Character.isLetter/isDigit has a valid keycode and that single-char input covers all Unicode; add early validation to reject unmappable or surrogate chars before touching the Robot: in typeCharacter check if Character.isSurrogate(character) (or Character.isHighSurrogate/isLowSurrogate) and throw IllegalArgumentException for supplementary code points, and after computing keyCode via KeyEvent.getExtendedKeyCodeForChar(...) ensure keyCode != KeyEvent.VK_UNDEFINED and throw before calling robot.keyPress; apply the same keyCode != VK_UNDEFINED guard for letter, digit and SPECIAL_CHAR_MAP branches (SPECIAL_CHAR_MAP entries should still be used when present), or alternatively change the API to accept a String and process code points with Character.codePointAt/Character.toChars and Character.toCodePoint for full Unicode support.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/EraseDataInFile.java-39-49 (1)
39-49:⚠️ Potential issue | 🟠 MajorSame null-safety issue – check
getValue()before callingtoString().If
filePath.getValue()returnsnull, line 39 throws an NPE before the validation on line 44 is reached.🐛 Proposed fix
try { - String path = filePath.getValue().toString(); - - logger.info("Erasing data from file: " + path); - - // Validate file path - if (path == null || path.trim().isEmpty()) { + Object value = filePath.getValue(); + if (value == null || value.toString().trim().isEmpty()) { setErrorMessage("File path cannot be empty"); ScreenshotUtils.captureAndUploadScreenshot(testStepResult, "erase_data_in_file_failure_screenshot", logger); return Result.FAILED; } + + String path = value.toString(); + logger.info("Erasing data from file: " + path);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/EraseDataInFile.java` around lines 39 - 49, The code calls filePath.getValue().toString() which can NPE if getValue() is null; first retrieve the value into a local (e.g., valueObj = filePath.getValue()), check valueObj for null or empty before calling toString(), and only then assign path = valueObj.toString(); if null/empty call setErrorMessage and ScreenshotUtils.captureAndUploadScreenshot(testStepResult, "erase_data_in_file_failure_screenshot", logger) and return Result.FAILED to preserve the existing error handling flow.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/AddDataToClipboard.java-37-47 (1)
37-47:⚠️ Potential issue | 🟠 MajorNull check is ineffective –
toString()will throw NPE before reaching it.On line 37,
dataToCopy.getValue().toString()is called. IfgetValue()returnsnull, this throws aNullPointerExceptionbefore the null check on line 42 is reached.🐛 Proposed fix to check for null before toString()
try { - String data = dataToCopy.getValue().toString(); - - logger.info("Adding data to clipboard: " + data); - - // Validate data - if (data == null) { + Object value = dataToCopy.getValue(); + if (value == null) { setErrorMessage("Data to copy cannot be null"); ScreenshotUtils.captureAndUploadScreenshot(testStepResult, "add_data_to_clipboard_failure_screenshot", logger); return Result.FAILED; } + + String data = value.toString(); + logger.info("Adding data to clipboard: " + data);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/AddDataToClipboard.java` around lines 37 - 47, The null check is after calling dataToCopy.getValue().toString(), which can NPE; fix by checking dataToCopy.getValue() for null before invoking toString(): first retrieve Object val = dataToCopy.getValue(), if val is null call setErrorMessage(...) and ScreenshotUtils.captureAndUploadScreenshot(testStepResult, "add_data_to_clipboard_failure_screenshot", logger) and return Result.FAILED; otherwise set String data = val.toString() and proceed with logger.info("Adding data to clipboard: " + data). Ensure you reference dataToCopy.getValue(), data, setErrorMessage, ScreenshotUtils.captureAndUploadScreenshot, logger, testStepResult and Result.FAILED in the updated flow.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnImage.java-249-429 (1)
249-429: 🛠️ Refactor suggestion | 🟠 MajorRemove large commented-out code block.
Lines 249-429 contain a large block of commented-out code (the previous OCR-based implementation). This should be removed to improve code clarity. If you need to reference the old implementation, it can be found in version control history.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnImage.java` around lines 249 - 429, Remove the large commented-out legacy OCR implementation inside the ClickOnImage class (the block that contains the old urlToFileConverter(...) and cleanupFile(...) implementations and the commented execute/closing braces); delete that entire commented section so only the active class code (current execute and any real helper methods) remains, and then clean up any now-unused imports or leftover comment markers to keep ClickOnImage and its real methods consistent.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnImage.java-174-177 (1)
174-177:⚠️ Potential issue | 🟠 MajorFormat string placeholders without arguments.
The error messages contain format placeholders (
%s) but no arguments are provided toString.format(). This will print literal%sin the error message.Suggested fix
} catch (IOException e) { - logger.info("Exception occurred while performing visual test at %s : %s" + ExceptionUtils.getStackTrace(e)); - setErrorMessage("Unable to perform visual testing for : <b> %s</b>"); - throw new RuntimeException("Error occurred while performing visual test at "); + logger.info("Exception occurred while performing visual test: " + ExceptionUtils.getStackTrace(e)); + setErrorMessage("Unable to perform visual testing"); + throw new RuntimeException("Error occurred while performing visual test", e);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnImage.java` around lines 174 - 177, The log and messages in ClickOnImage (catch block around the IOException) use "%s" placeholders but aren't passing arguments; update the logger.info, setErrorMessage, and the RuntimeException to use proper String.format(...) with the intended arguments (e.g., the image path or context and ExceptionUtils.getStackTrace(e) or include the exception as a parameter), or remove the "%s" placeholders and concatenate properly; ensure logger.info includes the exception details and setErrorMessage contains the target identifier, and throw a RuntimeException with a clear formatted message (use the same identifiers used in the original lines).windows_advanced_actions/src/main/java/com/testsigma/addons/util/OCRUtils.java-544-561 (1)
544-561:⚠️ Potential issue | 🟠 MajorOkHttp
Responseis not closed—potential resource leak.The
Responseobject fromclient.newCall(request).execute()must be closed to release the underlying connection. Even whenresponse.body().string()is called, the response should be closed in all code paths.🐛 Proposed fix using try-with-resources
logger.info("Making OCR API call"); - Response response = client.newCall(request).execute(); - - if (response.isSuccessful() && response.body() != null) { - String responseBody = response.body().string(); - logger.info("OCR Response body: " + responseBody); - - OCRResponse ocrResponse = mapper.readValue(responseBody, OCRResponse.class); - - if (ocrResponse.hasError()) { - throw new RuntimeException("OCR API returned error: " + ocrResponse.getError()); - } - if (!ocrResponse.hasText()) { - throw new RuntimeException("No text found in the image"); + try (Response response = client.newCall(request).execute()) { + if (response.isSuccessful() && response.body() != null) { + String responseBody = response.body().string(); + logger.info("OCR API call successful"); + + OCRResponse ocrResponse = mapper.readValue(responseBody, OCRResponse.class); + + if (ocrResponse.hasError()) { + throw new RuntimeException("OCR API returned error: " + ocrResponse.getError()); + } + if (!ocrResponse.hasText()) { + throw new RuntimeException("No text found in the image"); + } + return ocrResponse.getText(); + } else { + throw new RuntimeException("OCR API call failed with status: " + response.code()); } - return ocrResponse.getText(); - } else { - throw new RuntimeException("OCR API call failed with status: " + (response.code())); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/util/OCRUtils.java` around lines 544 - 561, The Response returned by client.newCall(request).execute() in OCRUtils (the block using Response response = client.newCall(request).execute()) is never closed, causing a resource leak; wrap the execute() call in a try-with-resources (try (Response response = client.newCall(request).execute()) { ... }) so the Response (and its body) is always closed on all code paths, keep the existing logic that reads response.body().string(), handle null body as before, and rethrow the same RuntimeExceptions inside the try block.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnPositionRelativeToText.java-222-247 (1)
222-247:⚠️ Potential issue | 🟠 MajorOkHttp Response not closed - potential resource leak.
The
Responseobject from OkHttp must be closed to release the underlying connection. Currently, if the response is successful, the connection may not be properly released.🐛 Proposed fix using try-with-resources
logger.info("Making OCR API call"); - Response response = client.newCall(request).execute(); - - if (response.isSuccessful()) { - logger.info("OCR API response is successful"); - if (response.body() != null) { - String responseBody = response.body().string(); + try (Response response = client.newCall(request).execute()) { + if (response.isSuccessful()) { + logger.info("OCR API response is successful"); + if (response.body() != null) { + String responseBody = response.body().string(); // ... rest of processing + } else { + throw new RuntimeException("OCR API returned null response body"); + } + } else { + throw new RuntimeException("OCR API call failed with status: " + response.code()); + } + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnPositionRelativeToText.java` around lines 222 - 247, The Response from OkHttp (variable Response response returned by client.newCall(request).execute() in ClickOnPositionRelativeToText) is not closed and can leak connections; wrap the execute call in a try-with-resources (e.g., try (Response response = client.newCall(request).execute()) { ... }) or explicitly close the response/body after reading, then read response.body().string() inside that scope, deserialize with mapper into ocrResponse, and keep existing error/status handling unchanged so the response is always closed even on exceptions.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnPositionRelativeToText.java-52-53 (1)
52-53: 🛠️ Refactor suggestion | 🟠 MajorInstance fields should be local variables.
ObjectMapperandOCRResponseare created as instance fields but used only withinexecute()andextractTextPoints(). TheocrResponsefield is particularly problematic as it gets overwritten on each OCR call, which could cause issues if this action is reused.♻️ Suggested refactor
- ObjectMapper mapper = new ObjectMapper(); - OCRResponse ocrResponse = new OCRResponse(); + private static final ObjectMapper mapper = new ObjectMapper();And make
ocrResponsea local variable insideextractTextPoints().🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnPositionRelativeToText.java` around lines 52 - 53, ObjectMapper and OCRResponse are declared as instance fields but only used inside execute() and extractTextPoints(), and ocrResponse is overwritten on each call; move their declarations into the methods that use them: create a local ObjectMapper inside execute() (or inside extractTextPoints() if only used there) and make ocrResponse a local variable inside extractTextPoints(), updating references in ClickOnPositionRelativeToText.execute() and ClickOnPositionRelativeToText.extractTextPoints() accordingly so no shared mutable state remains.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnTextWithWait.java-67-71 (1)
67-71:⚠️ Potential issue | 🟠 MajorHandle the same null OCR case before logging it.
getTextPointFromText()already treatsnullas a normal "not found yet" case, butprintAllCoordinates(textPoints)runs first and will throw on that same input. That turns a normal poll miss into an immediate failure.Also applies to: 137-153
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnTextWithWait.java` around lines 67 - 71, printAllCoordinates(textPoints) can NPE when OCR returns null, turning a normal "not found yet" poll into a hard failure; change the sequence in ClickOnTextWithWait so you call getTextPointFromText(textPoints) (or check textPoints for null/empty) before calling printAllCoordinates, and only invoke printAllCoordinates when textPoints is non-null and non-empty; apply the same defensive null/empty check-update to the other similar block that uses extractTextFromImage/getTextPointFromText/printAllCoordinates so both polling paths handle a null OCR result gracefully.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithSpacesAndWait.java-70-74 (1)
70-74:⚠️ Potential issue | 🟠 MajorEvery poll leaks a temporary screenshot.
screenshotFileis created on each iteration, but there is no cleanup on miss, success, or exceptions. Longer waits will keep leaving PNGs behind in the temp directory.Also applies to: 78-105, 123-135
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithSpacesAndWait.java` around lines 70 - 74, The loop in ClickOnTextWithSpacesAndWait creates a temporary screenshot via saveScreenshotToFile on every poll but never deletes it, leaking PNGs; update the polling logic (the method in class ClickOnTextWithSpacesAndWait that calls saveScreenshotToFile and extractTextPoints) to ensure each screenshotFile is deleted after use in all paths (miss, success, and exception) by moving deletion into a finally block or equivalent try-with-resources cleanup; apply the same cleanup to the other screenshot usages mentioned around the extractTextPoints call (the blocks covering lines ~78-105 and ~123-135) so every temporary file is removed regardless of outcome.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/WaitUntilTextPresentInScreen.java-81-88 (1)
81-88:⚠️ Potential issue | 🟠 MajorDon't stop polling before the declared timeout.
When
remainingTimedrops belowPOLLING_INTERVAL_MS, this breaks immediately instead of waiting out the remainder and doing one last OCR check. That makes the action fail up to 1.5 seconds early for texts that appear near the deadline.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/WaitUntilTextPresentInScreen.java` around lines 81 - 88, The loop exits early when remainingTime < POLLING_INTERVAL_MS, causing a missed final OCR attempt; in WaitUntilTextPresentInScreen replace the else { break } behavior by sleeping the remainingTime (if >0) and then performing one last check instead of breaking — use remainingTime = endTime - System.currentTimeMillis(); if (remainingTime > 0) Thread.sleep(remainingTime) (handle InterruptedException) and then proceed to run the final OCR check (or let the loop iterate once more) rather than breaking out; reference POLLING_INTERVAL_MS, remainingTime, endTime, Thread.sleep and the WaitUntilTextPresentInScreen polling loop.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnTextWithWait.java-90-98 (1)
90-98:⚠️ Potential issue | 🟠 MajorThe last sub-1.5s window is skipped.
As soon as
remainingTime <= POLLING_INTERVAL_MS, this breaks instead of waiting out the remainder and retrying once more. That shortens the requested wait and makes the action flaky near the deadline.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnTextWithWait.java` around lines 90 - 98, The loop in ClickOnTextWithWait uses endTime and POLLING_INTERVAL_MS and breaks when remainingTime <= POLLING_INTERVAL_MS, which skips the final partial wait; instead, compute remainingTime = endTime - System.currentTimeMillis() and if remainingTime > 0 sleep for Math.min(POLLING_INTERVAL_MS, remainingTime) (or sleep remainingTime if smaller) and then retry once more; only break when remainingTime <= 0 so the full requested timeout is honored.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/WaitUntilImagePresentWithFindImage.java-65-76 (1)
65-76:⚠️ Potential issue | 🟠 MajorThe polling loop never cleans up its temporary files.
Each iteration writes a new screenshot PNG, and
urlToFileConverter()may also create a temp copy for remoteimage-urlinputs. None of those files are deleted on success, timeout, or exception, so repeated waits will keep growing the temp directory. Be careful to only clean up files created by this action, not caller-supplied local paths.Also applies to: 79-90, 173-207
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/WaitUntilImagePresentWithFindImage.java` around lines 65 - 76, The polling loop creates temporary screenshot files and urlToFileConverter("target_image", imageUrlValue) may create a temp copy for remote inputs but none are deleted; update WaitUntilImagePresentWithFindImage to track which files it creates (e.g., the per-iteration screenshotFile and a boolean/marker for the searchImageFile returned by urlToFileConverter) and delete those files on success, timeout, and on any exception (use try/finally around the loop and ensure inside-loop cleanup of each screenshotFile after processing each iteration). Ensure you only delete files that this action created (do not delete caller-supplied local paths) by using the marker or by detecting remote-generated temp files from urlToFileConverter before removing them. Also apply the same cleanup pattern to the other affected blocks referenced (lines ~79-90 and ~173-207) where temp files are created.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/VerifyTextInApplication.java-17-31 (1)
17-31:⚠️ Potential issue | 🟠 MajorThe result runtime variable is declared but never set.
The action text says it stores the outcome in
result-variable-name, buttestData1is never assigned in either branch. Later steps cannot consume the verification result this action advertises.Also applies to: 59-66
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/VerifyTextInApplication.java` around lines 17 - 31, The runtime variable field testData1 (in class VerifyTextInApplication) is never assigned; update the action's execution flow (inside the execute method or the branches that determine text presence) to set the runtime variable represented by testData1 to the boolean result (e.g., "true"/"false") after the OCR check. Use the TestData runtime API on testData1 (e.g., call the TestData runtime setter provided by the Testsigma SDK such as setValue/setRuntimeValue or the project’s runtime variable helper) to persist the outcome so later steps can read the variable.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/VerifyTextInApplication.java-24-37 (1)
24-37:⚠️ Potential issue | 🟠 MajorThe advertised runtime variable is never populated.
The action text says the result is stored in
test-data1, buttestData1is never written before either return path. Downstream steps will always see that runtime variable unset.Also applies to: 93-101
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/VerifyTextInApplication.java` around lines 24 - 37, The runtime variable test-data1 is never assigned before the method returns; update the VerifyTextInApplication execution flow (e.g., in the execute() or equivalent method) to set testData1 with the result before every return path — for example call testData1.setValue(String.valueOf(found)) or testData1.setValue(foundText) depending on expected value and then proceed with the existing result handling; apply the same fix to the similar block mentioned around lines 93-101 so all exit paths write to testData1.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithSpacesAndWait.java-107-116 (1)
107-116:⚠️ Potential issue | 🟠 MajorThis wait can exit up to one poll early.
When
remainingTime <= POLLING_INTERVAL_MS, the loop breaks instead of waiting out the remainder and retrying once more. That shortens the advertised timeout and makes late-appearing text flaky.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithSpacesAndWait.java` around lines 107 - 116, The loop can exit up to one poll early because it breaks when remainingTime <= POLLING_INTERVAL_MS; instead, compute a sleep duration as the lesser of POLLING_INTERVAL_MS and remainingTime and sleep that amount if remainingTime > 0, only breaking when remainingTime <= 0. Update the block in ClickOnTextWithSpacesAndWait to use remainingTime/endTime and POLLING_INTERVAL_MS to derive nextSleep = Math.min(POLLING_INTERVAL_MS, remainingTime), log the actual sleep seconds, call Thread.sleep(nextSleep) when remainingTime > 0, and only break when remainingTime <= 0 so the full timeout is honored.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithSpacesAndWait.java-173-179 (1)
173-179:⚠️ Potential issue | 🟠 MajorTreat "no text in this frame" as a miss, not a failure.
In a wait action, an OCR response with no text should just mean "keep polling." Throwing here aborts the step on the first blank frame instead of waiting until timeout.
🛠️ Suggested change
- if (!ocrResponse.hasText()) { - throw new RuntimeException("No text found in the image"); - } + if (!ocrResponse.hasText()) { + logger.info("No OCR text detected in this poll attempt"); + return java.util.Collections.emptyList(); + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithSpacesAndWait.java` around lines 173 - 179, The current logic in ClickOnTextWithSpacesAndWait erroneously throws on ocrResponse.hasText() == false, causing the wait action to abort on the first blank frame; change the behavior so only ocrResponse.hasError() throws, while a missing text result is treated as a non-fatal miss (continue polling until timeout) by returning to the polling loop (or skipping this iteration) instead of throwing—update the code paths in ClickOnTextWithSpacesAndWait where ocrResponse is checked so that !ocrResponse.hasText() does not raise an exception but allows the wait loop to keep retrying.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToText.java-73-73 (1)
73-73:⚠️ Potential issue | 🟠 MajorAvoid logging raw OCR text from the entire screen.
printAllCoordinates()emits every recognized string and coordinate atinfo, which can leak unrelated UI content into step logs. Log the match target or element count instead.Also applies to: 217-220
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToText.java` at line 73, The call to printAllCoordinates(textPoints) in ClickOnPositionRelativeToText logs all OCR-recognized strings and coordinates and may leak unrelated UI content; replace these info-level dumps with minimal safe logs that record only the search target and counts (e.g., "Found N matches for '<targetText>'") and/or the chosen coordinate used for the click. Update both occurrences that call printAllCoordinates (the one shown and the other call around the 217-220 region) to stop printing raw OCR text and instead log the target string and textPoints.size() and the final selected Point.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/VerifyTextInApplicationWithAI.java-45-53 (1)
45-53:⚠️ Potential issue | 🟠 MajorClean up the temp screenshot after upload.
saveScreenshotToFile()creates a persistent temp PNG, but neither the success path nor the catch path removes it. Repeated steps will leave stale screenshots on the agent.Also applies to: 65-69, 73-83
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/VerifyTextInApplicationWithAI.java` around lines 45 - 53, The temp screenshot file created by saveScreenshotToFile in VerifyTextInApplicationWithAI is never deleted; after calling ScreenshotUtils.uploadScreenshotToS3 (and in any catch/failure branches) ensure the File returned by saveScreenshotToFile is deleted (or deletedOnExit) to avoid accumulating files. Update the method containing the snippet to wrap screenshot creation/processing in a try/finally (or ensure cleanup in both success and exception handlers) and call screenshotFile.delete() (or Files.deleteIfExists) after upload and after any early returns so that saveScreenshotToFile, OCRUtils.extractTextPoints, OCRUtils.searchForText, and ScreenshotUtils.uploadScreenshotToS3 all run but the temporary file is always removed afterward.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithWait.java-157-158 (1)
157-158:⚠️ Potential issue | 🟠 MajorRemove raw OCR text from logs.
Logging the full OCR response body and all detected strings dumps full-screen text into step logs, which is a privacy/compliance risk. Log counts or the matched string only.
Also applies to: 218-221
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithWait.java` around lines 157 - 158, The code currently logs the full OCR response body and all detected strings (see logger.info("OCR Response body: " + responseBody) in ClickOnTextWithWait and the similar logging at the later block around lines 218-221); remove these raw text dumps and replace them with privacy-safe logs that only record summary info such as the number of detected text blocks and, if needed, the single matched string (not the entire response). Update the logging calls that use responseBody and any collections of detected strings to log counts (e.g., detectedList.size()) and/or the matchedText variable only, and ensure no full OCR payload or full list of detected strings is written to logs.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToText.java-50-118 (1)
50-118:⚠️ Potential issue | 🟠 MajorHonor
wait-time-in-secondsin the execution path.
maxWaitSecondsis only used in log/error text here. The action does a single OCR pass and then fails immediately, so any text that appears after the first capture is missed.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToText.java` around lines 50 - 118, The current implementation in ClickOnPositionRelativeToText does a single OCR pass and ignores maxWaitSeconds; change the flow to poll until the timeout by wrapping the screen-capture→saveScreenshotToFile→ocr.extractTextFromImage→getTextPointFromText sequence in a loop that checks elapsed time against maxWaitSeconds (use System.currentTimeMillis() or similar), sleeping a short interval between attempts, and breaking when OCR returns a non-null OCRTextPoint; ensure calculateClickPosition, robot actions and the temporary screenshotFile cleanup happen only after a successful find, and always delete any temporary files created in each iteration to avoid leaks; keep existing log messages but move the timeout log/error and Result.FAILED return to after the polling loop so the action honors wait-time-in-seconds.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithWait.java-71-72 (1)
71-72:⚠️ Potential issue | 🟠 MajorDelete the temp screenshot on every polling iteration.
Each pass through the wait loop creates a new temp PNG and none of them are removed on retry, success, or exception. A long timeout will leak many files on the agent.
Also applies to: 97-119
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithWait.java` around lines 71 - 72, The per-iteration temp PNG created via saveScreenshotToFile(screenCapture, "click_text_screenshot") is never removed; modify the polling loop in ClickOnTextWithWait so each iteration deletes the temporary screenshot after use (including on success and on exceptions). Wrap the per-iteration screenshot creation/processing in a try/finally (or ensure a Files.deleteIfExists/screenshotFile.delete() is called) so the File returned from saveScreenshotToFile is removed after processing; apply the same cleanup for the other occurrences in the wait loop (the block covering the later saveScreenshotToFile calls around lines 97-119).windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToText.java-67-67 (1)
67-67:⚠️ Potential issue | 🟠 MajorDelete the temp screenshot from a
finallyblock.The current
delete()calls only run on the normal success/not-found branches. Any OCR or click failure aftersaveScreenshotToFile(...)leaks the PNG on the local agent.Also applies to: 98-109
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToText.java` at line 67, Declare and assign the File returned by saveScreenshotToFile(...) (the screenshotFile variable) outside/above the try block that performs OCR and clicking, and then move the deletion logic into a finally block so the temporary PNG is always removed even if OCR or click throws; specifically, ensure in the finally you check screenshotFile != null && screenshotFile.exists() and attempt screenshotFile.delete() (with optional logging on failure), and apply the same change to the other similar block referenced around lines 98-109 so both saveScreenshotToFile usages always clean up their temp files.windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithWait.java-163-169 (1)
163-169:⚠️ Potential issue | 🟠 MajorReturn an empty OCR result when nothing is detected.
Throwing here makes a blank/loading screen fail the step immediately, even though this action is supposed to keep polling until the wait time expires.
♻️ Suggested change
- if (!ocrResponse.hasText()) { - throw new RuntimeException("No text found in the image"); - } - - return ocrResponse.getText(); + if (!ocrResponse.hasText()) { + return java.util.Collections.emptyList(); + } + + return ocrResponse.getText();🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithWait.java` around lines 163 - 169, The code currently throws a RuntimeException when ocrResponse.hasText() is false, causing the step to fail immediately; modify ClickOnTextWithWait so that instead of throwing for missing text it returns an empty/blank OCR result (an OCRResponse instance with no text or a clearly empty marker) so the polling loop can continue until the wait expires; keep the existing throw for ocrResponse.hasError() as-is, and replace the `throw new RuntimeException("No text found in the image")` branch with a return/empty-response path that the surrounding polling logic in ClickOnTextWithWait can recognize and retry.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6675d7a8-875a-4144-905e-9af6f52f58ca
📒 Files selected for processing (56)
windows_advanced_actions/pom.xmlwindows_advanced_actions/src/main/java/com/testsigma/addons/util/Constants.javawindows_advanced_actions/src/main/java/com/testsigma/addons/util/KeyboardUtils.javawindows_advanced_actions/src/main/java/com/testsigma/addons/util/OCRResponse.javawindows_advanced_actions/src/main/java/com/testsigma/addons/util/OCRTextPoint.javawindows_advanced_actions/src/main/java/com/testsigma/addons/util/OCRUtils.javawindows_advanced_actions/src/main/java/com/testsigma/addons/util/ResponseObjectForFindImage.javawindows_advanced_actions/src/main/java/com/testsigma/addons/util/ScreenshotUtils.javawindows_advanced_actions/src/main/java/com/testsigma/addons/util/StringCompareUtil.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/AddDataToClipboard.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/AddDataToFile.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnFixedCoordinates.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnImage.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnImageWithFindImage.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnPositionRelativeToImage.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnPositionRelativeToText.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnRelativeCoordinates.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithSpacesAndWait.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ClickOnTextWithWait.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/CopyAndPasteTestDataOnScreen.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/EnterDataUsingKeyboard.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/EnterDataUsingKeyboardAndPressEnter.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/EraseDataInFile.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/ExtractDataFromScreen.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/PerformMultipleActions.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/PressFunctionKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/PressModifierKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/PressModifierKeyWithBasicKeyCombination.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/PressModifierKeyWithGivenKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/PressTwoModifierKeys.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/PressTwoModifierKeysWithBasicKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/StorePresenceOfTextInRuntimeVariable.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/StringCompareWindowsAdvanced.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/VerifyTextInApplication.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/VerifyTextInApplicationWithAI.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/WaitUntilImagePresent.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/WaitUntilImagePresentWithFindImage.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/WaitUntilTextPresentInScreen.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnImage.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToImage.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToText.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnTextWithWait.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/CopyAndPasteTestDataOnScreen.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/EnterDataUsingKeyboard.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/EnterDataUsingKeyboardAndPressEnter.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/PressFunctionKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/PressModifierKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/PressModifierKeyWithBasicKeyCombination.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/PressModifierKeyWithGivenKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/PressTwoModifierKeys.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/PressTwoModifierKeysWithBasicKey.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/RightClick.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/StorePresenceOfTextInRuntimeVariable.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/VerifyTextInApplication.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/WaitUntilImagePresent.javawindows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/WaitUntilTextPresentInScreen.java
| public static String VISUAL_SERVER_API_END_POINT = "https://visualtesting-staging.testsigma.com/image_analysis_with_files"; | ||
| public static String VISUAL_SERVER_FIND_IMAGE_ENDPOINT = "https://visualtesting-staging.testsigma.com/find_image_with_files"; | ||
| public static String VISUAL_SERVER_OCR_TEXT_ENDPOINT = "https://visualtesting-staging.testsigma.com/ocr-text-points-with-files"; | ||
|
|
||
|
|
||
|
|
||
| public static String API_TOKEN = "DHRNEYFTDCDGOEDDCOEICVYOEEUY"; |
There was a problem hiding this comment.
Move the visual-server config out of source control.
API_TOKEN is embedded in a public addon, and the endpoints are hard-wired to a staging host. That leaks a credential and makes rotation/promotion unsafe; load both from runtime config or a secret store instead.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/util/Constants.java`
around lines 4 - 10, Constants currently hard-code VISUAL_SERVER_API_END_POINT,
VISUAL_SERVER_FIND_IMAGE_ENDPOINT, VISUAL_SERVER_OCR_TEXT_ENDPOINT and API_TOKEN
in class Constants; remove these secrets/hosts from source and instead load them
at runtime (e.g., via environment variables, a configuration provider or secret
manager). Update Constants to read values from System.getenv()/a config helper
or inject a Config/Secrets service and fail fast if missing, referencing
VISUAL_SERVER_API_END_POINT, VISUAL_SERVER_FIND_IMAGE_ENDPOINT,
VISUAL_SERVER_OCR_TEXT_ENDPOINT and API_TOKEN so callers keep the same
identifiers while the actual values are sourced securely at runtime.
| if (operationResult) { | ||
| logger.info("Operation success: " + getSuccessMessage()); | ||
| setSuccessMessage(getSuccessMessage()); | ||
| result = com.testsigma.sdk.Result.SUCCESS; | ||
| } else { | ||
| logger.info("Operation failed: " + getErrorMessage()); | ||
| setErrorMessage(getErrorMessage()); | ||
| result = com.testsigma.sdk.Result.FAILED; | ||
| } |
There was a problem hiding this comment.
Critical bug: Success/error messages from utility are never used.
The code calls getSuccessMessage() and getErrorMessage() on this (the action class), not on the util object that computed the messages. This results in null or default messages being set instead of the meaningful messages generated by StringCompareUtil.
🐛 Proposed fix to use utility's messages
if (operationResult) {
- logger.info("Operation success: " + getSuccessMessage());
- setSuccessMessage(getSuccessMessage());
+ logger.info("Operation success: " + util.getSuccessMessage());
+ setSuccessMessage(util.getSuccessMessage());
result = com.testsigma.sdk.Result.SUCCESS;
} else {
- logger.info("Operation failed: " + getErrorMessage());
- setErrorMessage(getErrorMessage());
+ logger.info("Operation failed: " + util.getErrorMessage());
+ setErrorMessage(util.getErrorMessage());
result = com.testsigma.sdk.Result.FAILED;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (operationResult) { | |
| logger.info("Operation success: " + getSuccessMessage()); | |
| setSuccessMessage(getSuccessMessage()); | |
| result = com.testsigma.sdk.Result.SUCCESS; | |
| } else { | |
| logger.info("Operation failed: " + getErrorMessage()); | |
| setErrorMessage(getErrorMessage()); | |
| result = com.testsigma.sdk.Result.FAILED; | |
| } | |
| if (operationResult) { | |
| logger.info("Operation success: " + util.getSuccessMessage()); | |
| setSuccessMessage(util.getSuccessMessage()); | |
| result = com.testsigma.sdk.Result.SUCCESS; | |
| } else { | |
| logger.info("Operation failed: " + util.getErrorMessage()); | |
| setErrorMessage(util.getErrorMessage()); | |
| result = com.testsigma.sdk.Result.FAILED; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsAdvanced/StringCompareWindowsAdvanced.java`
around lines 45 - 53, The success/error messages are being read from the action
instance instead of the StringCompareUtil that computed them; update the logging
and message setters in StringCompareWindowsAdvanced (the branch where
operationResult is true/false) to use util.getSuccessMessage() and
util.getErrorMessage() (and pass those values to setSuccessMessage and
setErrorMessage and to logger.info) so the actual utility-generated messages are
logged and stored.
| cleanupFile(targetImageFile); | ||
| cleanupFile(baseImageFile); |
There was a problem hiding this comment.
Only clean up temp files you created.
For local inputs, urlToFileConverter() returns new File(url), and the success path always passes that file to cleanupFile(). A successful run therefore deletes the caller’s original image, while failure paths leak temp files because cleanup never runs from a finally block.
Also applies to: 163-166
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnImage.java`
around lines 94 - 95, The current cleanup calls (cleanupFile(targetImageFile);
cleanupFile(baseImageFile);) can delete caller-owned local files because
urlToFileConverter() returns new File(url) for local inputs; instead, track
which files you actually create (e.g., set boolean flags like targetImageIsTemp
and baseImageIsTemp when you download or create temp files) and move all cleanup
into a finally block; in that finally only call cleanupFile(...) for files where
the corresponding isTemp flag is true. Apply the same pattern to the second
cleanup site (the other cleanup calls around the same logic) so only temp files
created by ClickOnImage are deleted and all temp cleanup runs on both success
and failure.
| File targetImageFile = urlToFileConverter("target_image", imageUrlValue); | ||
| logger.info("Target image file prepared: " + targetImageFile.getAbsolutePath()); |
There was a problem hiding this comment.
Don't delete caller-supplied local image files.
For local image-url values, urlToFileConverter() returns new File(url), and both success/failure paths pass that file to cleanupFile(). That will remove the user's source image from disk. The converted file is also not used by ocr.findImage(), which still receives the original string.
Also applies to: 71-71, 102-113, 195-229
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/ClickOnPositionRelativeToImage.java`
around lines 52 - 53, The code currently calls
urlToFileConverter("target_image", imageUrlValue) and later always invokes
cleanupFile(...) which deletes caller-supplied local files because
urlToFileConverter returns new File(url) for local paths; change the cleanup
logic to only delete files that were actually created/downloaded by
urlToFileConverter (temporary files). Implement this by modifying
urlToFileConverter to return or expose a flag (or return a small holder)
indicating whether the File is temporary, or detect remote vs local input (e.g.,
URL scheme or temp directory), then call cleanupFile(targetImageFile) only when
that flag indicates ownership; ensure ocr.findImage(...) still receives the
correct original string where required and apply the same guarded cleanup to all
other usages noted (the blocks around targetImageFile at lines ~71, ~102-113,
and ~195-229).
| StringSelection stringSelection = new StringSelection(testData.getValue().toString()); | ||
| Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard(); | ||
| Transferable data = clipboard.getContents(null); | ||
| clipboard.setContents(stringSelection, null); | ||
| robot.keyPress(KeyEvent.VK_CONTROL); | ||
| robot.keyPress(KeyEvent.VK_V); | ||
| KeyboardUtils.sleep(100); | ||
| robot.keyRelease(KeyEvent.VK_V); | ||
| robot.keyRelease(KeyEvent.VK_CONTROL); | ||
| Thread.sleep(500); | ||
| clipboard.setContents(data, null); |
There was a problem hiding this comment.
Restore the clipboard in a finally block and handle a null return from getContents().
Clipboard.getContents(null) can return null, and any exception after modifying the clipboard leaves test data in the system clipboard. This pollutes the machine's clipboard and can cause subsequent paste operations to fail or expose temporary data. Clipboard restoration must occur in a finally block.
📋 Proposed fix
- Transferable data = clipboard.getContents(null);
- clipboard.setContents(stringSelection, null);
- robot.keyPress(KeyEvent.VK_CONTROL);
- robot.keyPress(KeyEvent.VK_V);
- KeyboardUtils.sleep(100);
- robot.keyRelease(KeyEvent.VK_V);
- robot.keyRelease(KeyEvent.VK_CONTROL);
- Thread.sleep(500);
- clipboard.setContents(data, null);
+ Transferable originalContents = clipboard.getContents(null);
+ clipboard.setContents(stringSelection, null);
+ try {
+ robot.keyPress(KeyEvent.VK_CONTROL);
+ robot.keyPress(KeyEvent.VK_V);
+ KeyboardUtils.sleep(100);
+ robot.keyRelease(KeyEvent.VK_V);
+ robot.keyRelease(KeyEvent.VK_CONTROL);
+ Thread.sleep(500);
+ } finally {
+ clipboard.setContents(
+ originalContents != null ? originalContents : new StringSelection(""),
+ null
+ );
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| StringSelection stringSelection = new StringSelection(testData.getValue().toString()); | |
| Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard(); | |
| Transferable data = clipboard.getContents(null); | |
| clipboard.setContents(stringSelection, null); | |
| robot.keyPress(KeyEvent.VK_CONTROL); | |
| robot.keyPress(KeyEvent.VK_V); | |
| KeyboardUtils.sleep(100); | |
| robot.keyRelease(KeyEvent.VK_V); | |
| robot.keyRelease(KeyEvent.VK_CONTROL); | |
| Thread.sleep(500); | |
| clipboard.setContents(data, null); | |
| StringSelection stringSelection = new StringSelection(testData.getValue().toString()); | |
| Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard(); | |
| Transferable originalContents = clipboard.getContents(null); | |
| clipboard.setContents(stringSelection, null); | |
| try { | |
| robot.keyPress(KeyEvent.VK_CONTROL); | |
| robot.keyPress(KeyEvent.VK_V); | |
| KeyboardUtils.sleep(100); | |
| robot.keyRelease(KeyEvent.VK_V); | |
| robot.keyRelease(KeyEvent.VK_CONTROL); | |
| Thread.sleep(500); | |
| } finally { | |
| clipboard.setContents( | |
| originalContents != null ? originalContents : new StringSelection(""), | |
| null | |
| ); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/CopyAndPasteTestDataOnScreen.java`
around lines 41 - 51, The clipboard manipulation in CopyAndPasteTestDataOnScreen
uses Clipboard.getContents(null) and then overwrites the clipboard without
guaranteeing restoration; wrap the clipboard replace + Robot key events
(StringSelection creation, clipboard.setContents(...), robot.keyPress/Release,
Thread.sleep) in a try block and move the clipboard.setContents(originalData,
null) into a finally block so the original Transferable (which may be null) is
restored after the operation; explicitly handle a null return from getContents()
when restoring (only call setContents if original is non-null or restore to an
empty/known safe value) and catch/log any IllegalStateException when calling
clipboard.setContents to avoid leaving the clipboard polluted.
| private boolean parseAIResponse(String aiResponse) { | ||
| if (aiResponse == null || aiResponse.trim().isEmpty()) { | ||
| logger.debug("AI response is null or empty"); | ||
| return false; | ||
| } | ||
|
|
||
| String response = aiResponse.trim().toUpperCase(); | ||
| logger.info("Parsing AI response: " + response); | ||
|
|
||
| // Check for various positive responses | ||
| if (response.contains("YES") || response.contains("TRUE") || response.contains("FOUND") || | ||
| response.contains("PRESENT") || response.contains("EXISTS")) { | ||
| return true; | ||
| } | ||
|
|
||
| // Check for various negative responses | ||
| if (response.contains("NO") || response.contains("FALSE") || response.contains("NOT FOUND") || | ||
| response.contains("ABSENT") || response.contains("NOT PRESENT")) { | ||
| return false; | ||
| } | ||
|
|
||
| // If response is unclear, log it and return false | ||
| logger.debug("Unclear AI response: " + aiResponse + ". Treating as 'not found'."); | ||
| return false; | ||
| } |
There was a problem hiding this comment.
Critical bug: Positive keywords match before negative keywords are checked.
The parsing logic checks for positive keywords first (lines 151-152), but several negative phrases contain positive substrings. For example:
"NOT FOUND"contains"FOUND"→ incorrectly returnstrue"NOT PRESENT"contains"PRESENT"→ incorrectly returnstrue"ABSENT"does not contain positive keywords, but the pattern is error-prone
The negative check (lines 157-158) is never reached when the response contains these phrases.
🐛 Proposed fix: Check negative phrases first
private boolean parseAIResponse(String aiResponse) {
if (aiResponse == null || aiResponse.trim().isEmpty()) {
logger.debug("AI response is null or empty");
return false;
}
String response = aiResponse.trim().toUpperCase();
logger.info("Parsing AI response: " + response);
- // Check for various positive responses
- if (response.contains("YES") || response.contains("TRUE") || response.contains("FOUND") ||
- response.contains("PRESENT") || response.contains("EXISTS")) {
- return true;
- }
-
// Check for various negative responses
- if (response.contains("NO") || response.contains("FALSE") || response.contains("NOT FOUND") ||
+ if (response.contains("NOT FOUND") || response.contains("NOT PRESENT") ||
response.contains("ABSENT") || response.contains("NOT PRESENT")) {
return false;
}
+
+ // Check for various positive responses
+ if (response.contains("YES") || response.contains("FOUND") ||
+ response.contains("PRESENT") || response.contains("EXISTS")) {
+ return true;
+ }
+
+ // Check simple NO/FALSE last to avoid matching "NOT" prefix issues
+ if (response.contains("NO") || response.contains("FALSE")) {
+ return false;
+ }
// If response is unclear, log it and return false
logger.debug("Unclear AI response: " + aiResponse + ". Treating as 'not found'.");
return false;
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/StorePresenceOfTextInRuntimeVariable.java`
around lines 141 - 165, In parseAIResponse, the positive keyword check runs
before negatives which causes phrases like "NOT FOUND"/"NOT PRESENT" to match
"FOUND"/"PRESENT" and return true; fix by evaluating negative phrases first
(check for "NO", "FALSE", "NOT FOUND", "NOT PRESENT", "ABSENT" against the
uppercased trimmed aiResponse) and returning false if any match, then evaluate
positive phrases and return true; keep existing null/empty guard (aiResponse ==
null || aiResponse.trim().isEmpty()) and logging (logger.debug/logger.info)
intact.
| // Check for various positive responses | ||
| if (response.contains("YES") || response.contains("TRUE") || response.contains("FOUND") || | ||
| response.contains("PRESENT") || response.contains("EXISTS")) { | ||
| return true; | ||
| } | ||
|
|
||
| // Check for various negative responses | ||
| if (response.contains("NO") || response.contains("FALSE") || response.contains("NOT FOUND") || | ||
| response.contains("ABSENT") || response.contains("NOT PRESENT")) { | ||
| return false; |
There was a problem hiding this comment.
Negative AI answers can be parsed as success.
"NOT FOUND" and "NOT PRESENT" both contain "FOUND"/"PRESENT", and the positive branch runs first. A response like No, the text was not found will incorrectly pass this step.
🐛 Suggested parser fix
- // Check for various positive responses
- if (response.contains("YES") || response.contains("TRUE") || response.contains("FOUND") ||
- response.contains("PRESENT") || response.contains("EXISTS")) {
- return true;
- }
-
- // Check for various negative responses
- if (response.contains("NO") || response.contains("FALSE") || response.contains("NOT FOUND") ||
- response.contains("ABSENT") || response.contains("NOT PRESENT")) {
+ if (response.contains("NOT FOUND") || response.contains("NOT PRESENT")
+ || response.contains("ABSENT") || response.startsWith("NO")
+ || response.startsWith("FALSE")) {
return false;
}
+
+ if (response.startsWith("YES") || response.startsWith("TRUE")
+ || response.contains("FOUND") || response.contains("PRESENT")
+ || response.contains("EXISTS")) {
+ return true;
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Check for various positive responses | |
| if (response.contains("YES") || response.contains("TRUE") || response.contains("FOUND") || | |
| response.contains("PRESENT") || response.contains("EXISTS")) { | |
| return true; | |
| } | |
| // Check for various negative responses | |
| if (response.contains("NO") || response.contains("FALSE") || response.contains("NOT FOUND") || | |
| response.contains("ABSENT") || response.contains("NOT PRESENT")) { | |
| return false; | |
| if (response.contains("NOT FOUND") || response.contains("NOT PRESENT") | |
| || response.contains("ABSENT") || response.startsWith("NO") | |
| || response.startsWith("FALSE")) { | |
| return false; | |
| } | |
| if (response.startsWith("YES") || response.startsWith("TRUE") | |
| || response.contains("FOUND") || response.contains("PRESENT") | |
| || response.contains("EXISTS")) { | |
| return true; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/VerifyTextInApplication.java`
around lines 147 - 156, The current contains-based checks in
VerifyTextInApplication treat "NOT FOUND"/"NOT PRESENT" as positives because
"FOUND"/"PRESENT" are matched first; modify the logic to either test negative
patterns before positives or, better, replace response.contains(...) checks with
whole-word/word-boundary matching (e.g., use Pattern/Matcher with regex like
"\bNOT FOUND\b", "\bNOT PRESENT\b", "\bFOUND\b", "\bPRESENT\b" against the
uppercased response variable) so "No, the text was not found" matches the
negative branch; update the if-blocks referencing response.contains(...)
accordingly.
| while (System.currentTimeMillis() < endTime) { | ||
| try { | ||
| // Fetch the Details of the Screen Size | ||
| Rectangle screenSize = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()); | ||
|
|
||
| // Take the Snapshot of the Screen | ||
| BufferedImage tmp = robot.createScreenCapture(screenSize); | ||
|
|
||
| // Provide the destination details to copy the screenshot | ||
| String tempDir = System.getProperty("java.io.tmpdir"); | ||
| String filename = "screenshot"+System.currentTimeMillis()+".jpg"; | ||
| String path = tempDir + filename; | ||
|
|
||
| // To copy source image in to destination path | ||
| ImageIO.write(tmp, "jpg",new File(path)); | ||
| int width = tmp.getWidth(); | ||
| int height = tmp.getHeight(); | ||
| logger.info("Width of image: " + width); | ||
| logger.info("Height of image: " + height); | ||
|
|
||
| File baseImageFile = new File(path); | ||
| String url = testStepResult.getScreenshotUrl(); | ||
| ocr.uploadFile(url, baseImageFile); | ||
| logger.info("url: "+ testStepResult.getScreenshotUrl()); | ||
| FindImageResponse responseObject = ocr.findImage(imageUrl.getValue().toString(), | ||
| Float.valueOf(threshold.getValue().toString())); | ||
| if (responseObject.getIsFound()){ | ||
| boolean isFound = responseObject.getIsFound(); | ||
| int x1 = responseObject.getX1(); | ||
| int y1 = responseObject.getY1(); | ||
| int x2 = responseObject.getX2(); | ||
| int y2 = responseObject.getY2(); | ||
|
|
||
| int clickLocationX = (x1 + x2) / 2; | ||
| int clickLocationY = (y1 + y2) / 2; | ||
|
|
||
| logger.info("Click Location X: " + clickLocationX); | ||
| logger.info("Click Location Y: " + clickLocationY); | ||
|
|
||
|
|
||
| setSuccessMessage("Image Found :" + isFound + | ||
| " Image coordinates :" + "x1-" + x1 + ", x2-" + x2 + ", y1-" + y1 + ", y2-" + y2); | ||
| Thread.sleep(1000); | ||
| return Result.SUCCESS; | ||
| } else { | ||
| setErrorMessage("Unable to fetch the coordinates"); | ||
| result = Result.FAILED; | ||
| return result; | ||
| } | ||
| } | ||
| catch (Exception e){ | ||
| logger.info("Exception: "+ ExceptionUtils.getStackTrace(e)); | ||
| setErrorMessage("Exception occurred while performing click action"); | ||
| result = Result.FAILED; | ||
| return result; | ||
| } | ||
| } |
There was a problem hiding this comment.
Polling logic is broken—loop exits on first OCR failure instead of retrying.
The loop immediately returns Result.FAILED (lines 111-114) when responseObject.getIsFound() is false, instead of sleeping and retrying until timeout. This defeats the purpose of the wait-until action.
Additionally:
POLLING_INTERVAL_MS(line 47) is never used—there's no sleep between iterations- The
searchImageFile(line 62) is created but never used;imageUrl.getValue().toString()is passed tofindImage()instead
🐛 Proposed fix for polling logic
if (responseObject.getIsFound()){
boolean isFound = responseObject.getIsFound();
int x1 = responseObject.getX1();
int y1 = responseObject.getY1();
int x2 = responseObject.getX2();
int y2 = responseObject.getY2();
int clickLocationX = (x1 + x2) / 2;
int clickLocationY = (y1 + y2) / 2;
logger.info("Click Location X: " + clickLocationX);
logger.info("Click Location Y: " + clickLocationY);
setSuccessMessage("Image Found :" + isFound +
" Image coordinates :" + "x1-" + x1 + ", x2-" + x2 + ", y1-" + y1 + ", y2-" + y2);
Thread.sleep(1000);
return Result.SUCCESS;
- } else {
- setErrorMessage("Unable to fetch the coordinates");
- result = Result.FAILED;
- return result;
}
+ // Image not found yet - continue polling
+ logger.info("Image not found yet, will retry...");
}
catch (Exception e){
logger.info("Exception: "+ ExceptionUtils.getStackTrace(e));
- setErrorMessage("Exception occurred while performing click action");
- result = Result.FAILED;
- return result;
+ // Log and continue polling on transient errors
+ logger.info("Transient error during polling, will retry...");
}
+
+ // Sleep before next polling attempt
+ long remainingTime = endTime - System.currentTimeMillis();
+ if (remainingTime > 0) {
+ Thread.sleep(Math.min(POLLING_INTERVAL_MS, remainingTime));
+ }
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@windows_advanced_actions/src/main/java/com/testsigma/addons/windowsLite/WaitUntilImagePresent.java`
around lines 67 - 123, The loop in WaitUntilImagePresent immediately returns
Result.FAILED when responseObject.getIsFound() is false instead of retrying;
change the logic so the method only returns SUCCESS when found or returns FAILED
after the timeout. Specifically, inside the while(System.currentTimeMillis() <
endTime) loop (the block that creates tmp, writes path, uploads via
ocr.uploadFile and calls ocr.findImage), remove the immediate
return/result-setting on a non-match; instead use
Thread.sleep(POLLING_INTERVAL_MS) between attempts (use the existing
POLLING_INTERVAL_MS constant) and continue looping. Also fix the unused
searchImageFile by passing the correct file/URL to ocr.findImage (use the
uploaded testStepResult.getScreenshotUrl() or the created searchImageFile path
consistently) and only set error messages/result = Result.FAILED after the loop
times out (set an appropriate timeout message). Ensure exception handling still
returns FAILED on unexpected errors.
please review this addon and publish as PUBLIC
Addon name : windows_advanced_actions
Addon accont: https://jarvis.testsigma.com/ui/tenants/2817/addons
updated windows advanced addon with OCR endpoints Instead Of Ai request
there are total 30 NLP's ( 1 action use Ai , Other actions use Robots and staging endpoints.
Summary by CodeRabbit