Skip to content

Screen-diff plugin should allow per-steps screenshots comparison #613

@sskorol

Description

@sskorol

I'm submitting a ...

  • bug report
  • feature request
  • support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Screen-diff plugin recognizes only test level attachments, and ignores made on steps level.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

@Attachment(value = "{name}", type = "image/png")
public static byte[] attachScreenshot(final String name, final byte[] data) {
    return data;
}

public static boolean hasDiff(final Screenshot actual, final Screenshot expected) {
    final ImageDiff diff = new ImageDiffer().makeDiff(expected, actual);
    final boolean hasDiff = diff.hasDiff();

    if (hasDiff) {
        addLabels(new Label().withName("testType").withValue("screenshotDiff"));
        attachScreenshot("actual", toByteArray(actual.getImage()));
        attachScreenshot("expected", toByteArray(expected.getImage()));
        attachScreenshot("diff", toByteArray(diff.getDiffImage()));
    }

    return hasDiff;
}

private static byte[] toByteArray(final BufferedImage image) {
    try (final ByteArrayOutputStream out = new ByteArrayOutputStream()) {
        ImageIO.write(image, "png", out);
        return out.toByteArray();
    } catch (IOException ignored) {
        return new byte[0];
    }
}

@Test
public void checkDiff() {
    assertThat(isSimilar("feature1", scr1, scr2)).isTrue();
    assertThat(isSimilar("feature2", scr3, scr4)).isTrue();
}

@Step("Compare screenshots for {name}.")
public boolean isSimilar(final String name, final Screenshot scr1, final Screenshot scr2) {
    return !hasDiff(scr1, scr2);
}

What is the expected behavior?

Each step should have individual screen-diff view, where user could check each compared pair.

What is the motivation / use case for changing the behavior?

For heavy UI tests it makes more sense to let users perform as many screen comparison operations they want, and avoid "only 1 pair per test" limitation.

Please tell us about your environment:

Allure version 2.3.5
Test framework testng@6.11
Allure adaptor allure-testng@2.0-BETA18
Generate report using allure-commandline@2.3.5

Other information

Seems like we just need to go deeper into steps hierarchy during attachments searching, instead of taking into account only test level screenshots.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions