diff --git a/.gitignore b/.gitignore index ad51f7fa8c08..960b438202cc 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,8 @@ bin/ *.jar *.war *.ear +# Except for illustrating jar file +!libraries-transform/src/main/resources/exampleforjadx/ExampleForJadx.jar # Eclipse @@ -29,6 +31,9 @@ out/ # Mac .DS_Store +# Quarkus +.quarkus/ + # Maven log/* target/ @@ -58,9 +63,9 @@ dependency-reduced-pom.xml *.dylib *.dll -xml/src/test/resources/example_dom4j_new.xml -xml/src/test/resources/example_dom4j_updated.xml -xml/src/test/resources/example_jaxb_new.xml +xml-modules/xml/src/test/resources/example_dom4j_new.xml +xml-modules/xml/src/test/resources/example_dom4j_updated.xml +xml-modules/xml/src/test/resources/example_jaxb_new.xml core-java-io/hard_link.txt core-java-io/target_link.txt core-java/src/main/java/com/baeldung/manifest/MANIFEST.MF @@ -96,7 +101,7 @@ customers.xml apache-cxf/cxf-aegis/baeldung.xml testing-modules/report-*.json -libraries-2/*.db +libraries-4/*.db apache-spark/data/output logs/ @@ -137,6 +142,7 @@ persistence-modules/neo4j/data/** /microservices-modules/micronaut-reactive/.micronaut/test-resources/test-resources.properties /libraries-security/src/main/resources/home/upload/test_file_SCP.txt /libraries-security/src/main/resources/home/upload/test_file_SFTP.txt +/libraries-security/decryptedFile /libraries-data-io/src/test/resources/protocols/gson_user.json /libraries-io/src/main/resources/application.csv /libraries-io/src/main/resources/application2.csv @@ -146,7 +152,7 @@ persistence-modules/neo4j/data/** /core-java-modules/core-java-io-conversions-3/src/test/resources/xlsxToCsv_output.csv /core-java-modules/core-java-io-5/output.txt /core-java-modules/core-java-io-apis-2/sample.txt -/persistence-modules/core-java-persistence-3/test.mv.db +/persistence-modules/core-java-persistence/test.mv.db /apache-libraries/src/main/java/com/baeldung/apache/avro/ /apache-libraries-2/cars.avro @@ -157,4 +163,9 @@ persistence-modules/neo4j/data/** /spring-cloud-modules/spring-cloud-bootstrap/gateway/src/main/resources/static/home/server/* /web-modules/linkrest/src/main/java/com/baeldung/cayenne/auto/_Department.java -/web-modules/linkrest/src/main/java/com/baeldung/cayenne/auto/_Employee.java \ No newline at end of file +/web-modules/linkrest/src/main/java/com/baeldung/cayenne/auto/_Employee.java + +#log4j +logging-modules/log4j/app-dynamic-log.log +logging-modules/logback/conditional.log +logging-modules/logback/filtered.log diff --git a/Jenkinsfile b/Jenkinsfile new file mode 100644 index 000000000000..1892d82cf18b --- /dev/null +++ b/Jenkinsfile @@ -0,0 +1,2 @@ +@Library('Jenkins-pipeline-lib-tutorials') _ +JavaTutorialsJenkinsfile() diff --git a/README.md b/README.md index a173a4d71a79..bf7b0d9dca48 100644 --- a/README.md +++ b/README.md @@ -36,13 +36,13 @@ Profile-based segregation We use Maven build profiles to segregate the huge list of individual projects in our repository. -The projects are broadly divided into 4 lists: default, default-jdk17, default-jdk8 and default-heavy. +The projects are broadly divided into 8 lists: default, default-jdk17, default-jdk22, default-jdk23, default-jdk24, default-jdk25, default-jdk8 and default-heavy. Next, they are segregated further based on the tests that we want to execute. We also have a parents profile to build only parent modules. -Therefore, we have a total of 9 profiles: +Therefore, we have a total of 17 profiles: | Profile | Includes | Type of test enabled | |-------------------|-----------------------------|----------------------| @@ -54,6 +54,10 @@ Therefore, we have a total of 9 profiles: | integration-jdk22 | JDK22 projects | *IntegrationTest | | default-jdk23 | JDK23 projects | *UnitTest | | integration-jdk23 | JDK23 projects | *IntegrationTest | +| default-jdk24 | JDK24 projects | *UnitTest | +| integration-jdk24 | JDK24 projects | *IntegrationTest | +| default-jdk25 | JDK25 projects | *UnitTest | +| integration-jdk25 | JDK25 projects | *IntegrationTest | | default-heavy | Heavy/long running projects | *UnitTest | | integration-heavy | Heavy/long running projects | *IntegrationTest | | default-jdk8 | JDK8 projects | *UnitTest | @@ -63,7 +67,7 @@ Therefore, we have a total of 9 profiles: Building the project ==================== -Though it should not be needed often to build the entire repository at once because we are usually concerned with a specific module. +Though it shouldn't be needed often to build the entire repository at once because we are usually concerned with a specific module. But if we want to, we can invoke the below command from the root of the repository if we want to build the entire repository with only Unit Tests enabled: diff --git a/akka-modules/README.md b/akka-modules/README.md deleted file mode 100644 index b85789407f1c..000000000000 --- a/akka-modules/README.md +++ /dev/null @@ -1,3 +0,0 @@ -## Akka - -This module contains modules about Akka. \ No newline at end of file diff --git a/akka-modules/akka-actors/README.md b/akka-modules/akka-actors/README.md deleted file mode 100644 index 20611aee6a4b..000000000000 --- a/akka-modules/akka-actors/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Akka HTTP - -This module contains articles about Akka actors. - -### Relevant articles: - -- [Introduction to Akka Actors in Java](https://www.baeldung.com/akka-actors-java) diff --git a/akka-modules/akka-http/README.md b/akka-modules/akka-http/README.md deleted file mode 100644 index ebe6581ff603..000000000000 --- a/akka-modules/akka-http/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Akka HTTP - -This module contains articles about Akka HTTP. - -### Relevant articles: - -- [Introduction to Akka HTTP](https://www.baeldung.com/akka-http) diff --git a/akka-modules/akka-http/pom.xml b/akka-modules/akka-http/pom.xml index 9372107fc9f2..8af72588cd5d 100644 --- a/akka-modules/akka-http/pom.xml +++ b/akka-modules/akka-http/pom.xml @@ -21,7 +21,7 @@ com.typesafe.akka akka-stream_${scala.version} - ${akka.stream.version} + ${akkastreams.version} com.typesafe.akka @@ -38,7 +38,6 @@ 10.0.11 - 2.5.11 \ No newline at end of file diff --git a/akka-modules/akka-http/src/test/java/com/baeldung/akkahttp/UserServerUnitTest.java b/akka-modules/akka-http/src/test/java/com/baeldung/akkahttp/UserServerUnitTest.java index 0bb9dc1ef258..432d2ba265b9 100644 --- a/akka-modules/akka-http/src/test/java/com/baeldung/akkahttp/UserServerUnitTest.java +++ b/akka-modules/akka-http/src/test/java/com/baeldung/akkahttp/UserServerUnitTest.java @@ -19,7 +19,6 @@ public class UserServerUnitTest extends JUnitRouteTest { TestRoute appRoute = testRoute(new UserServer(userActorRef).routes()); - @Ignore @Test public void whenRequest_thenActorResponds() { diff --git a/akka-modules/akka-streams/README.md b/akka-modules/akka-streams/README.md deleted file mode 100644 index a59b7fde5c3f..000000000000 --- a/akka-modules/akka-streams/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Akka Streams - -This module contains articles about Akka Streams. - -### Relevant articles - -- [Guide to Akka Streams](https://www.baeldung.com/akka-streams) diff --git a/akka-modules/akka-streams/pom.xml b/akka-modules/akka-streams/pom.xml index adf6e9b3d9ce..f9a1ead52a59 100644 --- a/akka-modules/akka-streams/pom.xml +++ b/akka-modules/akka-streams/pom.xml @@ -25,8 +25,4 @@ - - 2.5.2 - - \ No newline at end of file diff --git a/akka-modules/pom.xml b/akka-modules/pom.xml index be0b8aa89125..90c46da5bbbe 100644 --- a/akka-modules/pom.xml +++ b/akka-modules/pom.xml @@ -33,6 +33,7 @@ 2.12 + 2.5.11 \ No newline at end of file diff --git a/akka-modules/spring-akka/README.md b/akka-modules/spring-akka/README.md deleted file mode 100644 index c7db5d5c0134..000000000000 --- a/akka-modules/spring-akka/README.md +++ /dev/null @@ -1,6 +0,0 @@ -## Spring Akka - -This module contains articles about Spring with Akka - -### Relevant Articles: -- [Introduction to Spring with Akka](https://www.baeldung.com/akka-with-spring) diff --git a/akka-modules/spring-akka/pom.xml b/akka-modules/spring-akka/pom.xml index 67b3b1d6a28b..60c2082050c9 100644 --- a/akka-modules/spring-akka/pom.xml +++ b/akka-modules/spring-akka/pom.xml @@ -13,6 +13,18 @@ 1.0.0-SNAPSHOT + + + + org.springframework + spring-framework-bom + ${spring.version} + pom + import + + + + org.springframework @@ -30,18 +42,6 @@ - - - - org.springframework - spring-framework-bom - ${spring.version} - pom - import - - - - 6.1.5 2.4.17 diff --git a/algorithms-modules/algorithms-genetic/README.md b/algorithms-modules/algorithms-genetic/README.md deleted file mode 100644 index eb4e3fb798e8..000000000000 --- a/algorithms-modules/algorithms-genetic/README.md +++ /dev/null @@ -1,10 +0,0 @@ -## Genetic Algorithms - -This module contains articles about genetic algorithms. - -### Relevant articles: - -- [Introduction to Jenetics Library](https://www.baeldung.com/jenetics) -- [Ant Colony Optimization with a Java Example](https://www.baeldung.com/java-ant-colony-optimization) -- [Design a Genetic Algorithm in Java](https://www.baeldung.com/java-genetic-algorithm) -- [The Traveling Salesman Problem in Java](https://www.baeldung.com/java-simulated-annealing-for-traveling-salesman) diff --git a/algorithms-modules/algorithms-miscellaneous-1/README.md b/algorithms-modules/algorithms-miscellaneous-1/README.md deleted file mode 100644 index a3b6cbdaf831..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-1/README.md +++ /dev/null @@ -1,14 +0,0 @@ -## Algorithms - Miscellaneous - -This module contains articles about algorithms. Some classes of algorithms, e.g., [sorting](/../algorithms-sorting) and -[genetic algorithms](/../algorithms-genetic), have their own dedicated modules. - -### Relevant articles: - -- [Dijkstra Shortest Path Algorithm in Java](https://www.baeldung.com/java-dijkstra) -- [Implementing Simple State Machines with Java Enums](https://www.baeldung.com/java-enum-simple-state-machine) -- [Permutations of an Array in Java](https://www.baeldung.com/java-array-permutations) -- [Maximum Subarray Problem in Java](https://www.baeldung.com/java-maximum-subarray) -- [Converting Between Byte Arrays and Hexadecimal Strings in Java](https://www.baeldung.com/java-byte-arrays-hex-strings) -- [Calculate Distance Between Two Coordinates in Java](https://www.baeldung.com/java-find-distance-between-points) -- More articles: [[next -->]](/algorithms-miscellaneous-2) diff --git a/algorithms-modules/algorithms-miscellaneous-1/pom.xml b/algorithms-modules/algorithms-miscellaneous-1/pom.xml index 38ea383c32cd..bcadae04efc9 100644 --- a/algorithms-modules/algorithms-miscellaneous-1/pom.xml +++ b/algorithms-modules/algorithms-miscellaneous-1/pom.xml @@ -68,7 +68,6 @@ - 3.3.0 4.0.0 diff --git a/algorithms-modules/algorithms-miscellaneous-10/pom.xml b/algorithms-modules/algorithms-miscellaneous-10/pom.xml index 12c475f4a86d..0fa21df800b9 100644 --- a/algorithms-modules/algorithms-miscellaneous-10/pom.xml +++ b/algorithms-modules/algorithms-miscellaneous-10/pom.xml @@ -13,4 +13,4 @@ 1.0.0-SNAPSHOT - \ No newline at end of file + diff --git a/algorithms-modules/algorithms-miscellaneous-10/src/main/java/com/baeldung/algorithms/fizzbuzz/FizzBuzz.java b/algorithms-modules/algorithms-miscellaneous-10/src/main/java/com/baeldung/algorithms/fizzbuzz/FizzBuzz.java new file mode 100644 index 000000000000..4e3a8feb8936 --- /dev/null +++ b/algorithms-modules/algorithms-miscellaneous-10/src/main/java/com/baeldung/algorithms/fizzbuzz/FizzBuzz.java @@ -0,0 +1,93 @@ +package com.baeldung.algorithms.fizzbuzz; + +import java.util.ArrayList; +import java.util.List; + +/** + * FizzBuzz implementation that demonstrates three different approaches to solving + * the classic FizzBuzz programming puzzle. + *

+ * Problem Stmt: Given a positive integer n, iterate over 1 to n, print "Fizz" for multiples of 3, "Buzz" for + * multiples of 5, "FizzBuzz" for multiples of both, and the number otherwise. + */ +public class FizzBuzz { + + /** + * Naive approach using explicit modulo checks with an if-else chain. + * Order of conditions is critical here, so we must check divisibility + * by both 3 and 5 first. + * + * @param n positive integer (we iterate from 1 to n inclusive) + * @return FizzBuzz list with n elements + */ + public List fizzBuzzNaive(int n) { + List result = new ArrayList<>(); + for (int i = 1; i <= n; i++) { + if (i % 3 == 0 && i % 5 == 0) { + result.add("FizzBuzz"); + } else if (i % 3 == 0) { + result.add("Fizz"); + } else if (i % 5 == 0) { + result.add("Buzz"); + } else { + result.add(String.valueOf(i)); + } + } + return result; + } + + /** + * String concatenation approach that elegantly handles the FizzBuzz case. + * It uses StringBuilder and reuses the same object by setting its length to 0 + * using setLength(0) to avoid repeated instantiation. + * + * @param n positive integer (we iterate from 1 to n inclusive) + * @return FizzBuzz list with n elements + * @see Clearing StringBuilder + */ + public List fizzBuzzConcatenation(int n) { + List result = new ArrayList<>(); + StringBuilder output = new StringBuilder(); + for (int i = 1; i <= n; i++) { + if (i % 3 == 0) { + output.append("Fizz"); + } + if (i % 5 == 0) { + output.append("Buzz"); + } + result.add(output.length() > 0 ? output.toString() : String.valueOf(i)); + output.setLength(0); + } + return result; + } + + /** + * Counter approach that eliminates modulo operations using counters. + * It also uses StringBuilder and reuses the same object by settings + * its length to 0 using setLength(0) to avoid repeated instantiation. + * + * @param n positive integer (we iterate from 1 to n inclusive) + * @return FizzBuzz list with n elements + */ + public List fizzBuzzCounter(int n) { + List result = new ArrayList<>(); + StringBuilder output = new StringBuilder(); + int fizz = 0; + int buzz = 0; + for (int i = 1; i <= n; i++) { + fizz++; + buzz++; + if (fizz == 3) { + output.append("Fizz"); + fizz = 0; + } + if (buzz == 5) { + output.append("Buzz"); + buzz = 0; + } + result.add(output.length() > 0 ? output.toString() : String.valueOf(i)); + output.setLength(0); + } + return result; + } +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-miscellaneous-10/src/main/java/com/baeldung/algorithms/randomuniqueidentifier/UniqueIdGenerator.java b/algorithms-modules/algorithms-miscellaneous-10/src/main/java/com/baeldung/algorithms/randomuniqueidentifier/UniqueIdGenerator.java new file mode 100644 index 000000000000..e6c625103347 --- /dev/null +++ b/algorithms-modules/algorithms-miscellaneous-10/src/main/java/com/baeldung/algorithms/randomuniqueidentifier/UniqueIdGenerator.java @@ -0,0 +1,48 @@ +package com.baeldung.algorithms.randomuniqueidentifier; + +import java.security.SecureRandom; +import java.util.Set; +import java.util.stream.Collectors; + +/** + * Custom Random Identifier generator with unique ids . + */ +public class UniqueIdGenerator { + + private static final String ALPHANUMERIC = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"; + private static final SecureRandom random = new SecureRandom(); + + private int idLength = 8; // Default length + + /** + * Overrides the default ID length for generated identifiers. + * @param idLength The desired length (must be positive). + */ + public void setIdLength(int idLength) { + if (idLength <= 0) { + throw new IllegalArgumentException("Length must be positive"); + } + this.idLength = idLength; + } + + /** + * Generates a unique alphanumeric ID using the SecureRandom character mapping approach. + * @param existingIds A set of IDs already in use to ensure uniqueness. + * @return A unique alphanumeric string. + */ + public String generateUniqueId(Set existingIds) { + String newId; + do { + newId = generateRandomString(this.idLength); + } while (existingIds.contains(newId)); + return newId; + } + + private String generateRandomString(int length) { + return random.ints(length, 0, ALPHANUMERIC.length()) + .mapToObj(ALPHANUMERIC::charAt) + .map(Object::toString) + .collect(Collectors.joining()); + } +} + diff --git a/algorithms-modules/algorithms-miscellaneous-10/src/test/java/com/baeldung/algorithms/fizzbuzz/FizzBuzzUnitTest.java b/algorithms-modules/algorithms-miscellaneous-10/src/test/java/com/baeldung/algorithms/fizzbuzz/FizzBuzzUnitTest.java new file mode 100644 index 000000000000..1dbaf745f902 --- /dev/null +++ b/algorithms-modules/algorithms-miscellaneous-10/src/test/java/com/baeldung/algorithms/fizzbuzz/FizzBuzzUnitTest.java @@ -0,0 +1,73 @@ +package com.baeldung.algorithms.fizzbuzz; + +import java.util.List; +import java.util.stream.Collectors; +import java.util.stream.IntStream; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertAll; +import static org.junit.jupiter.api.Assertions.assertEquals; + +/** + * JUnit 5 test suite for FizzBuzz implementation. + * It tests all three approaches: Naive, Concatenation, and Counter. + *

+ * Test naming convention: givenWW_whenYY_thenXX + */ +class FizzBuzzUnitTest { + + private FizzBuzz fizzBuzz; + + private static final List GROUND_TRUTH_SEQUENCE_LENGTH_5 = generateGroundTruth(5); + + private static final List GROUND_TRUTH_SEQUENCE_LENGTH_100 = generateGroundTruth(100); + + @BeforeEach + void setUp() { + fizzBuzz = new FizzBuzz(); + } + + @Test + void givenSequenceLength5_whenAllMethods_thenReturnCorrectSequence() { + List naiveResult = fizzBuzz.fizzBuzzNaive(5); + List concatResult = fizzBuzz.fizzBuzzConcatenation(5); + List counterResult = fizzBuzz.fizzBuzzCounter(5); + + assertAll( + () -> assertEquals(GROUND_TRUTH_SEQUENCE_LENGTH_5, naiveResult, + "fizzBuzzNaive should return correct sequence for n=5"), + () -> assertEquals(GROUND_TRUTH_SEQUENCE_LENGTH_5, concatResult, + "fizzBuzzConcatenation should return correct sequence for n=5"), + () -> assertEquals(GROUND_TRUTH_SEQUENCE_LENGTH_5, counterResult, + "fizzBuzzCounter should return correct sequence for n=5") + ); + } + + @Test + void givenSequenceLength100_whenAllMethods_thenReturnCorrectSequence() { + List naiveResult = fizzBuzz.fizzBuzzNaive(100); + List concatResult = fizzBuzz.fizzBuzzConcatenation(100); + List counterResult = fizzBuzz.fizzBuzzCounter(100); + + assertAll( + () -> assertEquals(GROUND_TRUTH_SEQUENCE_LENGTH_100, naiveResult, + "fizzBuzzNaive should return correct sequence for n=100"), + () -> assertEquals(GROUND_TRUTH_SEQUENCE_LENGTH_100, concatResult, + "fizzBuzzConcatenation should return correct sequence for n=100"), + () -> assertEquals(GROUND_TRUTH_SEQUENCE_LENGTH_100, counterResult, + "fizzBuzzCounter should return correct sequence for n=100") + ); + } + + private static List generateGroundTruth(int n) { + return IntStream.rangeClosed(1, n) + .mapToObj(i -> { + if (i % 15 == 0) return "FizzBuzz"; + if (i % 3 == 0) return "Fizz"; + if (i % 5 == 0) return "Buzz"; + return String.valueOf(i); + }) + .collect(Collectors.toList()); + } +} diff --git a/algorithms-modules/algorithms-miscellaneous-10/src/test/java/com/baeldung/algorithms/randomuniqueidentifier/UniqueIdGeneratorUnitTest.java b/algorithms-modules/algorithms-miscellaneous-10/src/test/java/com/baeldung/algorithms/randomuniqueidentifier/UniqueIdGeneratorUnitTest.java new file mode 100644 index 000000000000..8413210be38e --- /dev/null +++ b/algorithms-modules/algorithms-miscellaneous-10/src/test/java/com/baeldung/algorithms/randomuniqueidentifier/UniqueIdGeneratorUnitTest.java @@ -0,0 +1,75 @@ +package com.baeldung.algorithms.randomuniqueidentifier; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import java.util.HashSet; +import java.util.Set; +import java.util.regex.Pattern; + +import static org.junit.jupiter.api.Assertions.*; + +class UniqueIdGeneratorUnitTest { + + private UniqueIdGenerator generator; + private static final Pattern ALPHANUMERIC_PATTERN = Pattern.compile("^[a-zA-Z0-9]+$"); + + @BeforeEach + void setUp() { + generator = new UniqueIdGenerator(); + } + + @Test + void givenDefaultSettings_whenGenerateUniqueIdIsCalled_thenReturnsValidAlphanumericString() { + Set existingIds = new HashSet<>(); + + String id = generator.generateUniqueId(existingIds); + + assertNotNull(id); + assertEquals(8, id.length()); + assertTrue(ALPHANUMERIC_PATTERN.matcher(id).matches()); + } + + @Test + void givenCustomLength_whenSetIdLengthIsCalled_thenGeneratorRespectsNewLength() { + generator.setIdLength(5); + Set existingIds = new HashSet<>(); + + String id = generator.generateUniqueId(existingIds); + + assertEquals(5, id.length()); + } + + @Test + void givenExistingId_whenGenerateUniqueIdIsCalled_thenReturnsNonCollidingId() { + // GIVEN: A set that already contains a specific ID + Set existingIds = new HashSet<>(); + String existingId = "ABC12345"; + existingIds.add(existingId); + + // WHEN: We generate a new ID + String newId = generator.generateUniqueId(existingIds); + + // THEN: The new ID must not match the existing one + assertNotEquals(existingId, newId); + } + + @Test + void givenLargeNumberRequests_whenGeneratedInBulk_thenAllIdsAreUnique() { + Set store = new HashSet<>(); + int count = 100; + + for (int i = 0; i < count; i++) { + store.add(generator.generateUniqueId(store)); + } + + assertEquals(count, store.size(), "All 100 generated IDs should be unique"); + } + + @Test + void givenInvalidLength_whenSetIdLengthIsCalled_thenThrowsException() { + assertThrows(IllegalArgumentException.class, () -> { + generator.setIdLength(0); + }); + } +} + diff --git a/algorithms-modules/algorithms-miscellaneous-2/README.md b/algorithms-modules/algorithms-miscellaneous-2/README.md deleted file mode 100644 index a51d786ae06f..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-2/README.md +++ /dev/null @@ -1,16 +0,0 @@ -## Algorithms - Miscellaneous - -This module contains articles about algorithms. Some classes of algorithms, e.g., [sorting](/../algorithms-sorting) and -[genetic algorithms](/../algorithms-genetic), have their own dedicated modules. - -### Relevant articles: - -- [Introduction to Cobertura](https://www.baeldung.com/cobertura) -- [Test a Linked List for Cyclicity](https://www.baeldung.com/java-linked-list-cyclicity) -- [Introduction to JGraphT](https://www.baeldung.com/jgrapht) -- [A Maze Solver in Java](https://www.baeldung.com/java-solve-maze) -- [Create a Sudoku Solver in Java](https://www.baeldung.com/java-sudoku) -- [Displaying Money Amounts in Words](https://www.baeldung.com/java-money-into-words) -- [A Collaborative Filtering Recommendation System in Java](https://www.baeldung.com/java-collaborative-filtering-recommendations) -- [Implementing A* Pathfinding in Java](https://www.baeldung.com/java-a-star-pathfinding) -- More articles: [[<-- prev]](/algorithms-miscellaneous-1) [[next -->]](/algorithms-miscellaneous-3) diff --git a/algorithms-modules/algorithms-miscellaneous-3/README.md b/algorithms-modules/algorithms-miscellaneous-3/README.md deleted file mode 100644 index 470ab0485291..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-3/README.md +++ /dev/null @@ -1,16 +0,0 @@ -## Algorithms - Miscellaneous - -This module contains articles about algorithms. Some classes of algorithms, e.g., [sorting](/algorithms-sorting) and -[genetic algorithms](/algorithms-genetic), have their own dedicated modules. - -## Relevant Articles: - -- [Java Two Pointer Technique](https://www.baeldung.com/java-two-pointer-technique) -- [Converting Between Roman and Arabic Numerals in Java](https://www.baeldung.com/java-convert-roman-arabic) -- [Checking If a List Is Sorted in Java](https://www.baeldung.com/java-check-if-list-sorted) -- [Checking if a Java Graph Has a Cycle](https://www.baeldung.com/java-graph-has-a-cycle) -- [A Guide to the Folding Technique in Java](https://www.baeldung.com/folding-hashing-technique) -- [Creating a Triangle with for Loops in Java](https://www.baeldung.com/java-print-triangle) -- [The K-Means Clustering Algorithm in Java](https://www.baeldung.com/java-k-means-clustering-algorithm) -- [Validating Input with Finite Automata in Java](https://www.baeldung.com/java-finite-automata) -- More articles: [[<-- prev]](/algorithms-miscellaneous-2) [[next -->]](/algorithms-miscellaneous-4) diff --git a/algorithms-modules/algorithms-miscellaneous-4/README.md b/algorithms-modules/algorithms-miscellaneous-4/README.md deleted file mode 100644 index 9b5b2c74018e..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-4/README.md +++ /dev/null @@ -1,15 +0,0 @@ -## Algorithms - Miscellaneous - -This module contains articles about algorithms. Some classes of algorithms, e.g., [sorting](https://github.com/eugenp/tutorials/blob/algorithms-sorting) and [genetic algorithms](https://github.com/eugenp/tutorials/blob/algorithms-genetic), have their own dedicated modules. - -### Relevant articles: - -- [Multi-Swarm Optimization Algorithm in Java](https://www.baeldung.com/java-multi-swarm-algorithm) -- [Check if a String Contains All the Letters of the Alphabet With Java](https://www.baeldung.com/java-string-contains-all-letters) -- [Find the Middle Element of a Linked List in Java](https://www.baeldung.com/java-linked-list-middle-element) -- [Find Substrings That Are Palindromes in Java](https://www.baeldung.com/java-palindrome-substrings) -- [Find the Longest Substring Without Repeating Characters](https://www.baeldung.com/java-longest-substring-without-repeated-characters) -- [Find the Smallest Missing Integer in an Array](https://www.baeldung.com/java-smallest-missing-integer-in-array) -- [Permutations of a String in Java](https://www.baeldung.com/java-string-permutations) -- [Example of Hill Climbing Algorithm in Java](https://www.baeldung.com/java-hill-climbing-algorithm) -- More articles: [[<-- prev]](/algorithms-miscellaneous-3) [[next -->]](/algorithms-miscellaneous-5) diff --git a/algorithms-modules/algorithms-miscellaneous-4/pom.xml b/algorithms-modules/algorithms-miscellaneous-4/pom.xml index 7ecf0ada089f..5a45478a25d5 100644 --- a/algorithms-modules/algorithms-miscellaneous-4/pom.xml +++ b/algorithms-modules/algorithms-miscellaneous-4/pom.xml @@ -37,8 +37,4 @@ - - 3.3.3 - - \ No newline at end of file diff --git a/algorithms-modules/algorithms-miscellaneous-5/README.md b/algorithms-modules/algorithms-miscellaneous-5/README.md deleted file mode 100644 index bb49680e4450..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-5/README.md +++ /dev/null @@ -1,17 +0,0 @@ -## Algorithms - Miscellaneous - -This module contains articles about algorithms. Some classes of algorithms, e.g., [sorting](/../algorithms-sorting) and -[genetic algorithms](/../algorithms-genetic), have their own dedicated modules. - -### Relevant articles: - -- [Reversing a Binary Tree in Java](https://www.baeldung.com/java-reversing-a-binary-tree) -- [Find If Two Numbers Are Relatively Prime in Java](https://www.baeldung.com/java-two-relatively-prime-numbers) -- [Knapsack Problem Implementation in Java](https://www.baeldung.com/java-knapsack) -- [How to Determine if a Binary Tree Is Balanced in Java](https://www.baeldung.com/java-balanced-binary-tree) -- [Overview of Combinatorial Problems in Java](https://www.baeldung.com/java-combinatorial-algorithms) -- [Prim’s Algorithm with a Java Implementation](https://www.baeldung.com/java-prim-algorithm) -- [How to Merge Two Sorted Arrays in Java](https://www.baeldung.com/java-merge-sorted-arrays) -- [Median of Stream of Integers using Heap in Java](https://www.baeldung.com/java-stream-integers-median-using-heap) -- More articles: [[<-- prev]](/algorithms-miscellaneous-4) [[next -->]](/algorithms-miscellaneous-6) - diff --git a/algorithms-modules/algorithms-miscellaneous-5/pom.xml b/algorithms-modules/algorithms-miscellaneous-5/pom.xml index 377e1d88a4cd..8c785ad0371e 100644 --- a/algorithms-modules/algorithms-miscellaneous-5/pom.xml +++ b/algorithms-modules/algorithms-miscellaneous-5/pom.xml @@ -7,7 +7,6 @@ 0.0.1-SNAPSHOT algorithms-miscellaneous-5 - com.baeldung algorithms-modules @@ -47,13 +46,10 @@ 17 17 + 17 - - 4.0.0 - - \ No newline at end of file diff --git a/algorithms-modules/algorithms-miscellaneous-6/README.md b/algorithms-modules/algorithms-miscellaneous-6/README.md deleted file mode 100644 index f21eddeed895..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-6/README.md +++ /dev/null @@ -1,13 +0,0 @@ -### Relevant Articles: - -- [Boruvka’s Algorithm for Minimum Spanning Trees in Java](https://www.baeldung.com/java-boruvka-algorithm) -- [Gradient Descent in Java](https://www.baeldung.com/java-gradient-descent) -- [Kruskal’s Algorithm for Spanning Trees with a Java Implementation](https://www.baeldung.com/java-spanning-trees-kruskal) -- [Balanced Brackets Algorithm in Java](https://www.baeldung.com/java-balanced-brackets-algorithm) -- [Efficiently Merge Sorted Java Sequences](https://www.baeldung.com/java-merge-sorted-sequences) -- [Introduction to Greedy Algorithms with Java](https://www.baeldung.com/java-greedy-algorithms) -- [The Caesar Cipher in Java](https://www.baeldung.com/java-caesar-cipher) -- [Implementing a 2048 Solver in Java](https://www.baeldung.com/2048-java-solver) -- [Finding Top K Elements in a Java Array](https://www.baeldung.com/java-array-top-elements) -- [Reversing a Linked List in Java](https://www.baeldung.com/java-reverse-linked-list) -- More articles: [[<-- prev]](/algorithms-miscellaneous-5) diff --git a/algorithms-modules/algorithms-miscellaneous-7/README.md b/algorithms-modules/algorithms-miscellaneous-7/README.md deleted file mode 100644 index 5de050525df7..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-7/README.md +++ /dev/null @@ -1,12 +0,0 @@ -### Relevant Articles: - -- [Algorithm to Identify and Validate a Credit Card Number](https://www.baeldung.com/java-validate-cc-number) -- [Find the N Most Frequent Elements in a Java Array](https://www.baeldung.com/java-n-most-frequent-elements-array) -- [Getting Pixel Array From Image in Java](https://www.baeldung.com/java-getting-pixel-array-from-image) -- [Rotate Arrays in Java](https://www.baeldung.com/java-rotate-arrays) -- [Find Missing Number From a Given Array in Java](https://www.baeldung.com/java-array-find-missing-number) -- [Calculate Weighted Mean in Java](https://www.baeldung.com/java-compute-weighted-average) -- [Check if Two Strings Are Rotations of Each Other](https://www.baeldung.com/java-string-check-strings-rotations) -- [Find the Largest Prime Under the Given Number in Java](https://www.baeldung.com/java-largest-prime-lower-threshold) -- [Count the Number of Unique Digits in an Integer using Java](https://www.baeldung.com/java-int-count-unique-digits) -- More articles: [[<-- prev]](/algorithms-miscellaneous-6) diff --git a/algorithms-modules/algorithms-miscellaneous-8/README.md b/algorithms-modules/algorithms-miscellaneous-8/README.md deleted file mode 100644 index 6c2af41f8f18..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-8/README.md +++ /dev/null @@ -1,12 +0,0 @@ -### Relevant Articles: -- [Vigenère Cipher in Java](https://www.baeldung.com/java-vigenere-cipher) -- [Merge Overlapping Intervals in a Java Collection](https://www.baeldung.com/java-collection-merge-overlapping-intervals) -- [Generate Juggler Sequence in Java](https://www.baeldung.com/java-generate-juggler-sequence) -- [Finding the Parent of a Node in a Binary Search Tree with Java](https://www.baeldung.com/java-find-parent-node-binary-search-tree) -- [Check if a Number Is a Happy Number in Java](https://www.baeldung.com/java-happy-sad-number-test) -- [Find the Largest Number Possible After Removing k Digits of a Number](https://www.baeldung.com/java-find-largest-number-remove-k-digits) -- [Implement Connect 4 Game with Java](https://www.baeldung.com/java-connect-4-game) -- [SkipList Implementation in Java](https://www.baeldung.com/java-skiplist) -- [Find the Date of Easter Sunday for the Given Year](https://www.baeldung.com/java-determine-easter-date-specific-year) -- [Calculating Moving Averages in Java](https://www.baeldung.com/java-compute-moving-average) -- More articles: [[<-- prev]](/algorithms-miscellaneous-7) diff --git a/algorithms-modules/algorithms-miscellaneous-8/pom.xml b/algorithms-modules/algorithms-miscellaneous-8/pom.xml index 086088114e92..67540f9fbd0d 100644 --- a/algorithms-modules/algorithms-miscellaneous-8/pom.xml +++ b/algorithms-modules/algorithms-miscellaneous-8/pom.xml @@ -20,4 +20,5 @@ ${commons-math3.version} + \ No newline at end of file diff --git a/algorithms-modules/algorithms-miscellaneous-9/README.md b/algorithms-modules/algorithms-miscellaneous-9/README.md deleted file mode 100644 index ec4c59de7766..000000000000 --- a/algorithms-modules/algorithms-miscellaneous-9/README.md +++ /dev/null @@ -1,9 +0,0 @@ -### Relevant Articles: -- [Check if Two Strings Are Permutations of Each Other in Java](https://www.baeldung.com/java-check-permutations-two-strings) -- [Finding the Mode of Integers in an Array in Java](https://www.baeldung.com/java-mode-integer-array) -- [Counting an Occurrence in an Array](https://www.baeldung.com/java-array-count-distinct-elements-frequencies) -- [Counting Subrrays Having a Specific Arithmetic Mean in Java](https://www.baeldung.com/java-count-subrrays-specified-average-value) -- [Finding the Closest Number to a Given Value From a List of Integers in Java](https://www.baeldung.com/java-list-find-closest-integer) -- [How to Find the Kth Largest Element in Java](https://www.baeldung.com/java-kth-largest-element) -- [Introduction to Minimax Algorithm with a Java Implementation](https://www.baeldung.com/java-minimax-algorithm) -- [How to Calculate Levenshtein Distance in Java?](https://www.baeldung.com/java-levenshtein-distance) \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/README.md b/algorithms-modules/algorithms-numeric/README.md deleted file mode 100644 index 89228f87cd5a..000000000000 --- a/algorithms-modules/algorithms-numeric/README.md +++ /dev/null @@ -1,4 +0,0 @@ -### Relevant Articles: - -- [How to Check Number Perfection](https://www.baeldung.com/java-number-perfection-test) -- [How to Check if a Number Is a Palindrome in Java](https://www.baeldung.com/java-palindrome-integer-test) diff --git a/algorithms-modules/algorithms-numeric/pom.xml b/algorithms-modules/algorithms-numeric/pom.xml index 4adcd2f2b398..968ce788a1b6 100644 --- a/algorithms-modules/algorithms-numeric/pom.xml +++ b/algorithms-modules/algorithms-numeric/pom.xml @@ -13,22 +13,33 @@ 1.0.0-SNAPSHOT - - 1.35 - - org.openjdk.jmh jmh-core ${jmh.version} - org.openjdk.jmh jmh-generator-annprocess ${jmh.version} + + org.nd4j + nd4j-api + ${nd4j.version} + + + org.nd4j + nd4j-native + ${nd4j.version} + runtime + + + 1.35 + 1.0.0-M2.1 + + \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/bcdtodecimal/BCDtoDecimalConverter.java b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/bcdtodecimal/BCDtoDecimalConverter.java new file mode 100644 index 000000000000..1549f79f9e50 --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/bcdtodecimal/BCDtoDecimalConverter.java @@ -0,0 +1,48 @@ +package com.baeldung.algorithms.bcdtodecimal; + +public class BCDtoDecimalConverter { + /** + * Converts a single packed BCD byte to an integer. + * Each byte represents two decimal digits. + * + * @param bcdByte The BCD byte to convert. + * @return The decimal integer value. + * @throws IllegalArgumentException if any nibble contains a non-BCD value (>9). + */ + public static int convertPackedByte(byte bcdByte) { + int resultDecimal; + int upperNibble = (bcdByte >> 4) & 0x0F; + int lowerNibble = bcdByte & 0x0F; + if (upperNibble > 9 || lowerNibble > 9) { + throw new IllegalArgumentException( + String.format("Invalid BCD format: byte 0x%02X contains non-decimal digit.", bcdByte) + ); + } + resultDecimal = upperNibble * 10 + lowerNibble; + return resultDecimal; + } + + /** + * Converts a BCD byte array to a long decimal value. + * Each byte in the array iis mapped to a packed BCD byte, + * representing two BCD nibbles. + * + * @param bcdArray The array of BCD bytes. + * @return The combined long decimal value. + * @throws IllegalArgumentException if any nibble contains a non-BCD value (>9). + */ + public static long convertPackedByteArray(byte[] bcdArray) { + long resultDecimal = 0; + for (byte bcd : bcdArray) { + int upperNibble = (bcd >> 4) & 0x0F; + int lowerNibble = bcd & 0x0F; + + if (upperNibble > 9 || lowerNibble > 9) { + throw new IllegalArgumentException("Invalid BCD format: nibble contains non-decimal digit."); + } + + resultDecimal = resultDecimal * 100 + (upperNibble * 10 + lowerNibble); + } + return resultDecimal; + } +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlur.java b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlur.java new file mode 100644 index 000000000000..42b2ef10ba2c --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlur.java @@ -0,0 +1,100 @@ +package com.baeldung.algorithms.fastgaussianblur; + +/** + * Fast Gaussian Blur approximation using sliding window over rows and columns + */ +public class FastGaussianBlur { + /** + * Horizontal box blur over rows + * + * @param source Source image row. + * @param target Target image row. + * @param width Image width. + * @param height Image height. + * @param radius Blur radius. + */ + private static void horizontalBoxBlur(int[] source, int[] target, int width, int height, int radius) { + double scale = 1.0 / (radius * 2 + 1); + + for (int y = 0; y < height; y++) { + int windowSum = 0; + int offset = y * width; + + // 1. We initialize the sliding window for the first pixel in the row + for (int x = -radius; x <= radius; x++) { + int safeX = Math.min(Math.max(x, 0), width - 1); + windowSum += source[offset + safeX]; + } + + // 2. We slide the window across the row + for (int x = 0; x < width; x++) { + target[offset + x] = (int) Math.round(windowSum * scale); + + // 2a. We subtract the leaving pixel and add the entering pixel + int leftX = Math.max(x - radius, 0); + int rightX = Math.min(x + radius + 1, width - 1); + + // 2b. We update the sliding window + windowSum -= source[offset + leftX]; + windowSum += source[offset + rightX]; + } + } + } + + /** + * Vertical box blur over columns. It is identical in logic, but we just + * traverse columns instead of rows + * + * @param source Source image row. + * @param target Target image row. + * @param width Image width. + * @param height Image height. + * @param radius Blur radius. + */ + private static void verticalBoxBlur(int[] source, int[] target, int width, int height, int radius) { + double scale = 1.0 / (radius * 2 + 1); + + for (int x = 0; x < width; x++) { + int windowSum = 0; + + for (int y = -radius; y <= radius; y++) { + int safeY = Math.min(Math.max(y, 0), height - 1); + windowSum += source[safeY * width + x]; + } + + for (int y = 0; y < height; y++) { + target[y * width + x] = (int) Math.round(windowSum * scale); + + int topY = Math.max(y - radius, 0); + int bottomY = Math.min(y + radius + 1, height - 1); + + windowSum -= source[topY * width + x]; + windowSum += source[bottomY * width + x]; + } + } + } + + /** + * Main orchestrator to call Fast Gaussian 2D Blur by separating it into + * two 1D operations + * @param source Source image row. + * @param width Image width + * @param height Image height + * @param radius Blur radius + * @param numPasses Number of Passes to Gaussian Approximate + */ + public static int[] applyFastGaussianBlur(int[] source, int width, int height, int radius, int numPasses) { + int[] target = new int[source.length]; + int[] temp = new int[source.length]; + + // 1. Copy original image data to target + System.arraycopy(source, 0, target, 0, source.length); + + // 2. Run (numPasses = 5) iterations for the Gaussian approximation + for (int i = 0; i < numPasses; i++) { + horizontalBoxBlur(target, temp, width, height, radius); + verticalBoxBlur(temp, target, width, height, radius); + } + return target; + } +} diff --git a/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlurRealImageTester.java b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlurRealImageTester.java new file mode 100644 index 000000000000..8e1d5a11f56c --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlurRealImageTester.java @@ -0,0 +1,89 @@ +package com.baeldung.algorithms.fastgaussianblur; + +import java.awt.image.BufferedImage; +import java.io.File; +import java.io.InputStream; +import javax.imageio.ImageIO; + +import javax.annotation.Nonnull; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Fast Gaussian Blur approximation tester on a real RGB image + */ +public class FastGaussianBlurRealImageTester { + + private static final Logger LOGGER = LoggerFactory.getLogger(FastGaussianBlurRealImageTester.class); + + @Nonnull + public static BufferedImage blurRealImage(@Nonnull BufferedImage image, int radius, int numPasses) { + int width = image.getWidth(); + int height = image.getHeight(); + + // 1. We extract 1D array of 32-bit ARGB pixels + int[] pixels = image.getRGB(0, 0, width, height, null, 0, width); + + // 2. We define arrays to hold independent color channels + int[] a = new int[pixels.length]; + int[] r = new int[pixels.length]; + int[] g = new int[pixels.length]; + int[] b = new int[pixels.length]; + + // 3. We unpack the 32-bit integers into separate channels + for (int i = 0; i < pixels.length; i++) { + a[i] = (pixels[i] >> 24) & 0xff; + r[i] = (pixels[i] >> 16) & 0xff; + g[i] = (pixels[i] >> 8) & 0xff; + b[i] = pixels[i] & 0xff; + } + + // 4. We apply our O(n) FastGaussianBlur algorithm to each color channel independently + r = FastGaussianBlur.applyFastGaussianBlur(r, width, height, radius, numPasses); + g = FastGaussianBlur.applyFastGaussianBlur(g, width, height, radius, numPasses); + b = FastGaussianBlur.applyFastGaussianBlur(b, width, height, radius, numPasses); + + // 5. We repack the channels back into a 32-bit integer array + int[] resultPixels = new int[pixels.length]; + for (int i = 0; i < pixels.length; i++) { + resultPixels[i] = (a[i] << 24) | (r[i] << 16) | (g[i] << 8) | b[i]; + } + + // 6. We create a new BufferedImage and set the blurred pixels + BufferedImage result = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB); + result.setRGB(0, 0, width, height, resultPixels, 0, width); + + // 7. We return the result + return result; + } + + public static void main(String[] args) throws Exception { + long startTime = System.currentTimeMillis(); + int radius = 1; + int numPasses = 5; + + // 1. We create a thread and read sapmple.jpg from ../src/man/resources + InputStream is = Thread.currentThread() + .getContextClassLoader() + .getResourceAsStream("sample.jpg"); + + if (is == null) { + throw new RuntimeException("Resource not found: sample.jpg"); + } + + BufferedImage originalImage = ImageIO.read(is); + assert originalImage != null; + + // 2. We run our Fast Blur algorithm + BufferedImage blurredImage = blurRealImage(originalImage, radius, numPasses); // Radius 10 + long endTime = System.currentTimeMillis(); + + LOGGER.debug("Blur completed in: " + (endTime - startTime) + " ms"); + + //3. We save result image sample_blurred.jpg under algorithms-numeric/target + File outputFile = new File("algorithms-numeric/target/sample_blurred.jpg"); + boolean status = ImageIO.write(blurredImage, "png", outputFile); + LOGGER.debug("Blur Operation: " + status + " Saved to: " + outputFile.getAbsolutePath()); + } +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/frogriverone/FrogRiverOne.java b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/frogriverone/FrogRiverOne.java new file mode 100644 index 000000000000..859c33dbcc3a --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/frogriverone/FrogRiverOne.java @@ -0,0 +1,51 @@ +/** + * Package to host code for Frog River One coding problem + */ + +package com.baeldung.algorithms.frogriverone; + +import java.util.HashSet; +import java.util.Set; +import javax.annotation.Nonnull; + +public class FrogRiverOne { + /* + * HashSet and Boolean array based solutions for Frog River One problem + * @param m Final destination + * @param leaves Integer array to hold leave positions + * + * @return status Integer to denote total time steps + * taken to reach destination + */ + public int HashSetSolution(int m, @Nonnull int[] leaves) { + Set leavesCovered = new HashSet<>(); + int status = -1; + for (int k = 0; k < leaves.length; k++) { + int position = leaves[k]; + leavesCovered.add(position); + + if (leavesCovered.size() == m) { + status = k + 1; + return status; + } + } + + return status; + } + + public int BooleanArraySolution(int m, @Nonnull int[] leaves) { + boolean[] leavesCovered = new boolean[m + 1]; + int leavesUncovered = m; + for (int k = 0; k< leaves.length; k++) { + int position = leaves[k]; + if (!leavesCovered[position]) { + leavesCovered[position] = true; + leavesUncovered--; + if (leavesUncovered == 0) { + return k + 1; + } + } + } + return -1; + } +} diff --git a/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/sockmerchant/SockMerchant.java b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/sockmerchant/SockMerchant.java new file mode 100644 index 000000000000..db184cebdce3 --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/sockmerchant/SockMerchant.java @@ -0,0 +1,33 @@ +package com.baeldung.algorithms.sockmerchant; + +import java.util.HashSet; +import java.util.Set; +import javax.annotation.Nonnull; + +public class SockMerchant { + public int countPairsWithArray(int n, @Nonnull int[] socks, int k) { + int[] counts = new int[k]; + int pairs = 0; + for (int i = 0; i < n; i++) { + counts[socks[i]]++; + } + for (int count : counts) { + pairs += count / 2; + } + return pairs; + } + + public int countPairsWithSet(@Nonnull int[] socks) { + Set unmatchedSocks = new HashSet<>(); + int pairs = 0; + for (int sock : socks) { + if (unmatchedSocks.contains(sock)) { + pairs++; + unmatchedSocks.remove(sock); + } else { + unmatchedSocks.add(sock); + } + } + return pairs; + } +} diff --git a/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/sumoftwosquares/NumberAsSumOfTwoSquares.java b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/sumoftwosquares/NumberAsSumOfTwoSquares.java new file mode 100644 index 000000000000..2dbf9848efaa --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/sumoftwosquares/NumberAsSumOfTwoSquares.java @@ -0,0 +1,59 @@ +package com.baeldung.algorithms.sumoftwosquares; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class NumberAsSumOfTwoSquares { + + private static final Logger LOGGER = LoggerFactory.getLogger(NumberAsSumOfTwoSquares.class); + + /** + * Checks if a non-negative integer n can be written as the + * sum of two squares i.e. (a^2 + b^2) + * This implementation is based on Fermat's theorem on sums of two squares. + * + * @param n The number to check (must be non-negative). + * @return true if n can be written as a sum of two squares, false otherwise. + */ + public static boolean isSumOfTwoSquares(int n) { + if (n < 0) { + LOGGER.warn("Input must be non-negative. Returning false for n = {}", n); + return false; + } + if (n == 0) { + return true; // 0 = 0^2 + 0^2 + } + + // 1. Reduce n to an odd number if n is even. + while (n % 2 == 0) { + n /= 2; + } + + // 2. Iterate through odd prime factors starting from 3 + for (int i = 3; i * i <= n; i += 2) { + // 2a. Find the exponent of the factor i + int count = 0; + while (n % i == 0) { + count++; + n /= i; + } + + // 2b. Check the condition from Fermat's theorem + // If i is of form 4k+3 (i % 4 == 3) and has an odd exponent + if (i % 4 == 3 && count % 2 != 0) { + LOGGER.debug("Failing condition: factor {} (form 4k+3) has odd exponent {}", i, count); + return false; + } + } + + // 3. Handle the last remaining factor (which is prime if > 1) + // If n itself is a prime of the form 4k+3, its exponent is 1 (odd). + if (n % 4 == 3) { + LOGGER.debug("Failing condition: remaining factor {} is of form 4k+3", n); + return false; + } + + // 4. All 4k+3 primes had even exponents. + return true; + } +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/twoanglesdifference/AngleDifferenceCalculator.java b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/twoanglesdifference/AngleDifferenceCalculator.java new file mode 100644 index 000000000000..0edd5d31f377 --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/main/java/com/baeldung/algorithms/twoanglesdifference/AngleDifferenceCalculator.java @@ -0,0 +1,64 @@ +/** + * Package to host code for calculating three types of angle difference + */ +package com.baeldung.algorithms.twoanglesdifference; + +public class AngleDifferenceCalculator { + + /** + * Normalizes an angle to be within the range [0, 360). + * + * @param angle The angle in degrees. + * @return The normalized angle. + */ + public static double normalizeAngle(double angle) { + return (angle % 360 + 360) % 360; + } + + /** + * Calculates the absolute difference between two angles. + * + * @param angle1 The first angle in degrees. + * @param angle2 The second angle in degrees. + * @return The absolute difference in degrees. + */ + public static double absoluteDifference(double angle1, double angle2) { + return Math.abs(angle1 - angle2); + } + + /** + * Calculates the shortest difference between two angles. + * + * @param angle1 The first angle in degrees. + * @param angle2 The second angle in degrees. + * @return The shortest difference in degrees (0 to 180). + */ + public static double shortestDifference(double angle1, double angle2) { + double diff = absoluteDifference(normalizeAngle(angle1), normalizeAngle(angle2)); + return Math.min(diff, 360 - diff); + } + + /** + * Calculates the signed shortest difference between two angles. + * A positive result indicates counter-clockwise rotation, a negative result indicates clockwise. + * + * @param angle1 The first angle in degrees. + * @param angle2 The second angle in degrees. + * @return The signed shortest difference in degrees (-180 to 180). + */ + public static double signedShortestDifference(double angle1, double angle2) { + double normalizedAngle1 = normalizeAngle(angle1); + double normalizedAngle2 = normalizeAngle(angle2); + double diff = normalizedAngle2 - normalizedAngle1; + + if (diff > 180) { + return diff - 360; + } else if (diff < -180) { + return diff + 360; + } else { + return diff; + } + } +} + + diff --git a/algorithms-modules/algorithms-numeric/src/main/resources/sample.jpg b/algorithms-modules/algorithms-numeric/src/main/resources/sample.jpg new file mode 100644 index 000000000000..e7c7b10812c5 Binary files /dev/null and b/algorithms-modules/algorithms-numeric/src/main/resources/sample.jpg differ diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/bcdtodecimal/BCDtoDecimalConverterTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/bcdtodecimal/BCDtoDecimalConverterTest.java new file mode 100644 index 000000000000..71fbf161c9fe --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/bcdtodecimal/BCDtoDecimalConverterTest.java @@ -0,0 +1,95 @@ +package com.baeldung.algorithms.bcdtodecimal; + +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class BCDtoDecimalConverterTest { + + // 1. Tests for convertPackedByte(byte bcdByte) + + @Test + void testConvertPackedByteValidValues() { + // Test 05 (0x05) -> + assertEquals(5, BCDtoDecimalConverter.convertPackedByte((byte) 0x05)); + + // Test 22 (0x22) -> 22 + assertEquals(22, BCDtoDecimalConverter.convertPackedByte((byte) 0x22)); + + // Test 97 (0x97) -> 97 + assertEquals(97, BCDtoDecimalConverter.convertPackedByte((byte) 0x097)); + } + + @Test + void testConvertPackedByteInvalidUpperNibbleThrowsException() { + // Test Upper nibble is A (1010), Lower nibble is 1 (0001) -> 0xA1 + byte invalidByte = (byte) 0xA1; + assertThrows(IllegalArgumentException.class, () -> BCDtoDecimalConverter.convertPackedByte(invalidByte), + "Received non-BCD upper nibble (A). Provide valid BCD nibbles (0-9)."); + } + + @Test + void testConvertPackedByteBothInvalidThrowsException() { + // test Upper nibble is B, Lower nibble is E -> 0xBE + byte invalidByte = (byte) 0xBE; + assertThrows(IllegalArgumentException.class, + () -> BCDtoDecimalConverter.convertPackedByte(invalidByte), + "Received both nibbles as non-BCD. Provide valid BCD nibbles (0-9)." + ); + } + + // ------------------------------------------------------------------------- + + // 2. Tests for convertPackedByteArray(byte[] bcdArray) + + @Test + void testConvertPackedByteArrayValidValues() { + // Test 0 -> [0x00] + assertEquals(0L, BCDtoDecimalConverter.convertPackedByteArray(new byte[]{(byte) 0x00})); + + // Test 99 -> [0x99] + assertEquals(99L, BCDtoDecimalConverter.convertPackedByteArray(new byte[]{(byte) 0x99})); + + // Test 1234 -> [0x12, 0x34] + byte[] bcd1234 = {(byte) 0x12, (byte) 0x34}; + assertEquals(1234L, BCDtoDecimalConverter.convertPackedByteArray(bcd1234)); + + // Test 12345678 -> [0x12, 0x34, 0x56, 0x78] + byte[] bcdLarge = {(byte) 0x12, (byte) 0x34, (byte) 0x56, (byte) 0x78}; + assertEquals(12345678L, BCDtoDecimalConverter.convertPackedByteArray(bcdLarge)); + } + + @Test + void testConvertPackedByteArrayEmptyArray() { + // Test empty array -> 0 + assertEquals(0L, BCDtoDecimalConverter.convertPackedByteArray(new byte[]{})); + } + + @Test + void testConvertPackedByteArrayMaximumSafeLong() { + // Test a large number that fits within a long (18 digits) + // 999,999,999,999,999,999 (18 nines) + byte[] bcdMax = {(byte) 0x99, (byte) 0x99, (byte) 0x99, (byte) 0x99, (byte) 0x99, (byte) 0x99, (byte) 0x99, (byte) 0x99, (byte) 0x99}; + assertEquals(999999999999999999L, BCDtoDecimalConverter.convertPackedByteArray(bcdMax)); + } + + @Test + void testConvertPackedByteArrayInvalidNibbleThrowsException() { + // Contains 0x1A (A is an invalid BCD digit) + byte[] bcdInvalid = {(byte) 0x12, (byte) 0x1A, (byte) 0x34}; + assertThrows(IllegalArgumentException.class, + () -> BCDtoDecimalConverter.convertPackedByteArray(bcdInvalid), + "Received array containing an invalid BCD byte. Provide valid BCD nibbles (0-9)." + ); + } + + @Test + void testConvertPackedByteArray_InvalidFirstByteThrowsException() { + // Invalid BCD byte at the start + byte[] bcdInvalid = {(byte) 0xF0, (byte) 0x12}; + assertThrows(IllegalArgumentException.class, + () -> BCDtoDecimalConverter.convertPackedByteArray(bcdInvalid), + "Received first byte as an invalid BCD byte. Provide valid BCD nibbles (0-9)." + ); + } +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/consecutivenumbers/ConsecutiveNumbersUnitTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/consecutivenumbers/ConsecutiveNumbersUnitTest.java new file mode 100644 index 000000000000..110b8c4beabe --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/consecutivenumbers/ConsecutiveNumbersUnitTest.java @@ -0,0 +1,31 @@ +package com.baeldung.algorithms.consecutivenumbers; + +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class ConsecutiveNumbersUnitTest { + + @Test + void whenIsSumOfConsecutiveUsingBruteForce_thenReturnsTrue() { + int n = 15; + + boolean isSumOfConsecutive = false; + for (int k = 2; (k * (k - 1)) / 2 < n; k++) { + int diff = n - k * (k - 1) / 2; + if (diff % k == 0 && diff / k > 0) { + isSumOfConsecutive = true; + break; + } + } + + assertTrue(isSumOfConsecutive); + } + + @Test + void whenIsSumOfConsecutiveUsingBitwise_thenReturnsTrue() { + int n = 15; + boolean result = (n > 0) && ((n & (n - 1)) != 0); + assertTrue(result); + } + +} diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/cosinesimilarity/CosineSimilarityUnitTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/cosinesimilarity/CosineSimilarityUnitTest.java new file mode 100644 index 000000000000..ec7fe65a50bb --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/cosinesimilarity/CosineSimilarityUnitTest.java @@ -0,0 +1,80 @@ +package com.baeldung.algorithms.cosinesimilarity; + +import org.junit.jupiter.api.Test; +import org.nd4j.linalg.api.ndarray.INDArray; +import org.nd4j.linalg.api.ops.impl.reduce3.CosineSimilarity; +import org.nd4j.linalg.factory.Nd4j; + +import java.util.Arrays; +import java.util.stream.IntStream; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +public class CosineSimilarityUnitTest { + + static final double[] VECTOR_A = {3, 4}; + static final double[] VECTOR_B = {5, 12}; + static final double EXPECTED_SIMILARITY = 0.9692307692307692; + + static double calculateCosineSimilarity(double[] vectorA, double[] vectorB) { + if (vectorA == null || vectorB == null || vectorA.length != vectorB.length || vectorA.length == 0) { + throw new IllegalArgumentException("Vectors must be non-null, non-empty, and of the same length."); + } + double dotProduct = 0.0; + double magnitudeA = 0.0; + double magnitudeB = 0.0; + for (int i = 0; i < vectorA.length; i++) { + dotProduct += vectorA[i] * vectorB[i]; + magnitudeA += vectorA[i] * vectorA[i]; + magnitudeB += vectorB[i] * vectorB[i]; + } + double finalMagnitudeA = Math.sqrt(magnitudeA); + double finalMagnitudeB = Math.sqrt(magnitudeB); + if (finalMagnitudeA == 0.0 || finalMagnitudeB == 0.0) { + return 0.0; + } + return dotProduct / (finalMagnitudeA * finalMagnitudeB); + } + + public static double calculateCosineSimilarityWithStreams(double[] vectorA, double[] vectorB) { + if (vectorA == null || vectorB == null || vectorA.length != vectorB.length || vectorA.length == 0) { + throw new IllegalArgumentException("Vectors must be non-null, non-empty, and of the same length."); + } + + double dotProduct = IntStream.range(0, vectorA.length).mapToDouble(i -> vectorA[i] * vectorB[i]).sum(); + double magnitudeA = Arrays.stream(vectorA).map(v -> v * v).sum(); + double magnitudeB = IntStream.range(0, vectorA.length).mapToDouble(i -> vectorB[i] * vectorB[i]).sum(); + double finalMagnitudeA = Math.sqrt(magnitudeA); + double finalMagnitudeB = Math.sqrt(magnitudeB); + if (finalMagnitudeA == 0.0 || finalMagnitudeB == 0.0) { + return 0.0; + } + + return dotProduct / (finalMagnitudeA * finalMagnitudeB); + } + + @Test + void givenTwoHighlySimilarVectors_whenCalculatedNatively_thenReturnsHighSimilarityScore() { + double actualSimilarity = calculateCosineSimilarity(VECTOR_A, VECTOR_B); + assertEquals(EXPECTED_SIMILARITY, actualSimilarity, 1e-15); + } + + @Test + void givenTwoHighlySimilarVectors_whenCalculatedNativelyWithStreams_thenReturnsHighSimilarityScore() { + double actualSimilarity = calculateCosineSimilarityWithStreams(VECTOR_A, VECTOR_B); + assertEquals(EXPECTED_SIMILARITY, actualSimilarity, 1e-15); + } + + @Test + void givenTwoHighlySimilarVectors_whenCalculatedNativelyWithCommonsMath_thenReturnsHighSimilarityScore() { + + INDArray vec1 = Nd4j.create(VECTOR_A); + INDArray vec2 = Nd4j.create(VECTOR_B); + + CosineSimilarity cosSim = new CosineSimilarity(vec1, vec2); + double actualSimilarity = Nd4j.getExecutioner().exec(cosSim).getDouble(0); + + assertEquals(EXPECTED_SIMILARITY, actualSimilarity, 1e-15); + } + +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlurUnitTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlurUnitTest.java new file mode 100644 index 000000000000..aa8c6ac83c42 --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/fastgaussianblur/FastGaussianBlurUnitTest.java @@ -0,0 +1,32 @@ +package com.baeldung.algorithms.fastgaussianblur; +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertTrue; + + +/** + * Fast Gaussian Blur JUnit Test case + */ +class FastGaussianBlurUnitTest { + @Test + void givenSharpImage_whenAppliedBlur_thenCenterIsSmoothed() { + int width = 5; + int height = 5; + int numPasses = 5; + int[] image = new int[width * height]; + + // 1. We create an impulse image with all indices black image except one bright white pixel in the center. + image[12] = 255; + + int[] blurredImage = FastGaussianBlur.applyFastGaussianBlur(image, width, height, 1, numPasses); + + // 2. We expect the center pixel should lose intensity + // as it gets spread to its neighbors + assertTrue(blurredImage[12] < 255); + assertTrue(blurredImage[12] > 0); + + // 3. We expect that its immediate neighbors should have + // gained intensity + assertTrue(blurredImage[11] > 0); // Left neighbor + assertTrue(blurredImage[13] > 0); // Right neighbor + } +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/frogriverone/FrogRiverOneUnitTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/frogriverone/FrogRiverOneUnitTest.java new file mode 100644 index 000000000000..f9e6a2092fcb --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/frogriverone/FrogRiverOneUnitTest.java @@ -0,0 +1,39 @@ +/** + * Package to host JUNIT5 Unit Test code for Frog River One coding problem + */ + +package com.baeldung.algorithms.frogriverone; + +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertEquals; + +class FrogRiverOneUnitTest { + + private final FrogRiverOne frogRiverOne = new FrogRiverOne(); + + @Test + void whenLeavesCoverPath_thenReturnsEarliestTime() { + int m = 7; + int[] leaves = {1, 3, 6, 4, 2, 3, 7, 5, 4}; + // Expected: Time 8 (Value 5 falls at index 7, completing 1..7) + + // HashSet based solution + assertEquals(8, frogRiverOne.HashSetSolution(m, leaves)); + + // Boolean array based solution + assertEquals(8, frogRiverOne.BooleanArraySolution(m, leaves)); + } + + @Test + void whenLeavesAreMissing_thenReturnsMinusOne() { + int m = 7; + int[] leaves = {1, 3, 6, 4, 2, 3, 7, 4}; //missing 5 + + // HashSet based Solution + assertEquals(-1, frogRiverOne.HashSetSolution(m, leaves)); + + // Boolean array based solution + assertEquals(-1, frogRiverOne.BooleanArraySolution(m, leaves)); + } + +} diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/sockmerchant/SockMerchantUnitTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/sockmerchant/SockMerchantUnitTest.java new file mode 100644 index 000000000000..e9bd732cc024 --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/sockmerchant/SockMerchantUnitTest.java @@ -0,0 +1,25 @@ +package com.baeldung.algorithms.sockmerchant; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import java.util.Arrays; + +import org.junit.Test; + +public class SockMerchantUnitTest { + @Test + public void givenSockArray_whenUsingArray_thenReturnsCorrectPairCount() { + SockMerchant merchant = new SockMerchant(); + int[] socks = {11, 22, 22, 11, 33, 3, 33, 111111, 33, 222222}; + int colorMax = 222223; + int actualPairs = merchant.countPairsWithArray(socks.length, socks, colorMax); + assertEquals(3, actualPairs); + } + + @Test + public void givenSockArray_whenUsingSet_thenReturnsCorrectPairCount() { + SockMerchant merchant = new SockMerchant(); + int[] socks = {11, 22, 22, 11, 33, 3, 33, 111111, 33, 222222}; + int actualPairs = merchant.countPairsWithSet(socks); + assertEquals(3, actualPairs); + } +} diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/sumoftwosquares/NumberAsSumOfTwoSquaresUnitTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/sumoftwosquares/NumberAsSumOfTwoSquaresUnitTest.java new file mode 100644 index 000000000000..ecf2f35efa9b --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/sumoftwosquares/NumberAsSumOfTwoSquaresUnitTest.java @@ -0,0 +1,47 @@ +package com.baeldung.algorithms.sumoftwosquares; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.DisplayName; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.assertFalse; + + +class NumberAsSumOfTwoSquaresUnitTest { + + @Test + @DisplayName("Given input number can be expressed as a sum of squares, when checked, then returns true") + void givenNumberIsSumOfSquares_whenCheckIsCalled_thenReturnsTrue() { + // Simple cases + assertTrue(NumberAsSumOfTwoSquares.isSumOfTwoSquares(0)); // 0^2 + 0^2 + assertTrue(NumberAsSumOfTwoSquares.isSumOfTwoSquares(1)); // 1^2 + 0^2 + assertTrue(NumberAsSumOfTwoSquares.isSumOfTwoSquares(5)); // 1^2 + 2^2 + assertTrue(NumberAsSumOfTwoSquares.isSumOfTwoSquares(8)); // 2^2 + 2^2 + + // Cases from Fermat theorem + assertTrue(NumberAsSumOfTwoSquares.isSumOfTwoSquares(50)); // 2 * 5^2. No 4k+3 primes. + assertTrue(NumberAsSumOfTwoSquares.isSumOfTwoSquares(45)); // 3^2 * 5. 4k+3 prime (3) has even exp. + assertTrue(NumberAsSumOfTwoSquares.isSumOfTwoSquares(18)); // 2 * 3^2. 4k+3 prime (3) has even exp. + } + + @Test + @DisplayName("Given input number can't be expressed as a sum of squares, when checked, then returns false") + void givenNumberIsNotSumOfSquares_whenCheckIsCalled_thenReturnsFalse() { + // Simple cases + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(3)); // 3 (4k+3, exp 1) + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(6)); // 2 * 3 (3 has exp 1) + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(7)); // 7 (4k+3, exp 1) + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(11)); // 11 (4k+3, exp 1) + + // Cases from theorem + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(12)); // 2^2 * 3 (3 has exp 1) + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(21)); // 3 * 7 (both 3 and 7 have exp 1) + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(28)); // 2^2 * 7 (7 has exp 1) + } + + @Test + @DisplayName("Given input number is negative, when checked, then returns false") + void givenNegativeNumber_whenCheckIsCalled_thenReturnsFalse() { + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(-1)); // Negatives as hygiene + assertFalse(NumberAsSumOfTwoSquares.isSumOfTwoSquares(-10)); // Negatives as hygiene + } +} diff --git a/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/twoanglesdifference/AngleDifferenceCalculatorTest.java b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/twoanglesdifference/AngleDifferenceCalculatorTest.java new file mode 100644 index 000000000000..1539c6b9e37b --- /dev/null +++ b/algorithms-modules/algorithms-numeric/src/test/java/com/baeldung/algorithms/twoanglesdifference/AngleDifferenceCalculatorTest.java @@ -0,0 +1,47 @@ +/** + * Package to host JUnit Test code for AngleDifferenceCalculator Class + */ +package com.baeldung.algorithms.twoanglesdifference; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +import org.junit.jupiter.api.Test; + + + +class AngleDifferenceCalculatorTest { + + private static final double EPSILON = 0.0001; + + @Test + void whenNormalizingAngle_thenReturnsCorrectRange() { + assertEquals(90, AngleDifferenceCalculator.normalizeAngle(450), EPSILON); + assertEquals(30, AngleDifferenceCalculator.normalizeAngle(390), EPSILON); + assertEquals(330, AngleDifferenceCalculator.normalizeAngle(-30), EPSILON); + assertEquals(0, AngleDifferenceCalculator.normalizeAngle(360), EPSILON); + } + + @Test + void whenCalculatingAbsoluteDifference_thenReturnsCorrectValue() { + assertEquals(100, AngleDifferenceCalculator.absoluteDifference(10, 110), EPSILON); + assertEquals(290, AngleDifferenceCalculator.absoluteDifference(10, 300), EPSILON); + assertEquals(30, AngleDifferenceCalculator.absoluteDifference(-30, 0), EPSILON); + } + + @Test + void whenCalculatingShortestDifference_thenReturnsCorrectValue() { + assertEquals(100, AngleDifferenceCalculator.shortestDifference(10, 110), EPSILON); + assertEquals(70, AngleDifferenceCalculator.shortestDifference(10, 300), EPSILON); + assertEquals(30, AngleDifferenceCalculator.shortestDifference(-30, 0), EPSILON); + assertEquals(0, AngleDifferenceCalculator.shortestDifference(360, 0), EPSILON); + } + + @Test + void whenCalculatingSignedShortestDifference_thenReturnsCorrectValue() { + assertEquals(100, AngleDifferenceCalculator.signedShortestDifference(10, 110), EPSILON); + assertEquals(-70, AngleDifferenceCalculator.signedShortestDifference(10, 300), EPSILON); + assertEquals(30, AngleDifferenceCalculator.signedShortestDifference(-30, 0), EPSILON); + assertEquals(70, AngleDifferenceCalculator.signedShortestDifference(300, 10), EPSILON); + } +} + diff --git a/algorithms-modules/algorithms-optimization/.gitignore b/algorithms-modules/algorithms-optimization/.gitignore new file mode 100644 index 000000000000..30b2b7442c55 --- /dev/null +++ b/algorithms-modules/algorithms-optimization/.gitignore @@ -0,0 +1,4 @@ +/target/ +.settings/ +.classpath +.project \ No newline at end of file diff --git a/algorithms-modules/algorithms-optimization/pom.xml b/algorithms-modules/algorithms-optimization/pom.xml new file mode 100644 index 000000000000..e6d1f07e70e1 --- /dev/null +++ b/algorithms-modules/algorithms-optimization/pom.xml @@ -0,0 +1,58 @@ + + + 4.0.0 + algorithms-optimization + 0.0.1-SNAPSHOT + algorithms-optimization + + + com.baeldung + algorithms-modules + 1.0.0-SNAPSHOT + + + + + org.apache.commons + commons-math3 + ${commons-math3.version} + + + commons-codec + commons-codec + ${commons-codec.version} + + + org.projectlombok + lombok + ${lombok.version} + provided + + + org.ojalgo + ojalgo + ${ojalgo.version} + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.11.0 + + 17 + + + + + + + 3.6.1 + 56.2.0 + + + \ No newline at end of file diff --git a/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/AssignmentSolution.java b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/AssignmentSolution.java new file mode 100644 index 000000000000..b1bd86d07f0f --- /dev/null +++ b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/AssignmentSolution.java @@ -0,0 +1,47 @@ +package com.baeldung.algorithms.optimization.lp; + +public class AssignmentSolution { + + private double totalCost; + private double[][] assignment; + + public AssignmentSolution(double totalCost, double[][] assignment) { + this.totalCost = totalCost; + this.assignment = assignment; + } + + public double getTotalCost() { + return totalCost; + } + + public void setTotalCost(double totalCost) { + this.totalCost = totalCost; + } + + public double[][] getAssignment() { + return assignment; + } + + public void setAssignment(double[][] assignment) { + this.assignment = assignment; + } + + @Override + public String toString() { + StringBuilder sb = new StringBuilder(); + sb.append("AssignmentSolution\n"); + sb.append("Total Cost: ").append(totalCost).append("\n"); + sb.append("Assignment Matrix:\n"); + for (int i = 0; i < assignment.length; i++) { + sb.append("[ "); + for (int j = 0; j < assignment[i].length; j++) { + sb.append(String.format("%.0f", assignment[i][j])); + if (j < assignment[i].length - 1) { + sb.append(", "); + } + } + sb.append(" ]\n"); + } + return sb.toString(); + } +} diff --git a/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/AssignmentSolver.java b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/AssignmentSolver.java new file mode 100644 index 000000000000..86422b398208 --- /dev/null +++ b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/AssignmentSolver.java @@ -0,0 +1,7 @@ +package com.baeldung.algorithms.optimization.lp; + +public interface AssignmentSolver { + + AssignmentSolution solve(double[][] cost); + +} diff --git a/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/CommonsMathAssignmentSolver.java b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/CommonsMathAssignmentSolver.java new file mode 100644 index 000000000000..a7f354a05666 --- /dev/null +++ b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/CommonsMathAssignmentSolver.java @@ -0,0 +1,80 @@ +package com.baeldung.algorithms.optimization.lp; + +import java.util.ArrayList; +import java.util.Collection; + +import org.apache.commons.math3.optim.PointValuePair; +import org.apache.commons.math3.optim.linear.LinearConstraint; +import org.apache.commons.math3.optim.linear.LinearConstraintSet; +import org.apache.commons.math3.optim.linear.LinearObjectiveFunction; +import org.apache.commons.math3.optim.linear.NonNegativeConstraint; +import org.apache.commons.math3.optim.linear.Relationship; +import org.apache.commons.math3.optim.linear.SimplexSolver; +import org.apache.commons.math3.optim.nonlinear.scalar.GoalType; + +public class CommonsMathAssignmentSolver implements AssignmentSolver { + + public AssignmentSolution solve(double[][] t) { + + int volunteers = t.length; + int locations = t[0].length; + int vars = volunteers * locations; + + // Objective function coefficients + double[] x = new double[vars]; + for (int i = 0; i < volunteers; i++) { + for (int j = 0; j < locations; j++) { + x[index(i, j, locations)] = t[i][j]; + } + } + + LinearObjectiveFunction objective = new LinearObjectiveFunction(x, 0); + + Collection constraints = new ArrayList<>(); + + // Each volunteer assigned to exactly one location + for (int i = 0; i < volunteers; i++) { + double[] x_i = new double[vars]; + for (int j = 0; j < locations; j++) { + x_i[index(i, j, locations)] = 1.0; + } + constraints.add(new LinearConstraint(x_i, Relationship.EQ, 1.0)); + } + + // Each location gets exactly one volunteer + for (int j = 0; j < locations; j++) { + double[] x_j = new double[vars]; + for (int i = 0; i < volunteers; i++) { + x_j[index(i, j, locations)] = 1.0; + } + constraints.add(new LinearConstraint(x_j, Relationship.EQ, 1.0)); + } + + // Solve LP + SimplexSolver solver = new SimplexSolver(); + PointValuePair solution = solver.optimize( + objective, + new LinearConstraintSet(constraints), + GoalType.MINIMIZE, + new NonNegativeConstraint(true) + ); + + double totalCost = solution.getValue(); + double[] point = solution.getPoint(); + + // Rebuild assignment matrix + double[][] assignment = new double[volunteers][locations]; + for (int i = 0; i < volunteers; i++) { + for (int j = 0; j < locations; j++) { + assignment[i][j] = point[index(i, j, locations)]; + } + } + + return new AssignmentSolution(totalCost, assignment); + } + + private int index(int i, int j, int locations) { + return i * locations + j; + } + +} diff --git a/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/OjAlgoAssignmentSolver.java b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/OjAlgoAssignmentSolver.java new file mode 100644 index 000000000000..1097e9cae786 --- /dev/null +++ b/algorithms-modules/algorithms-optimization/src/main/java/com/baeldung/algorithms/optimization/lp/OjAlgoAssignmentSolver.java @@ -0,0 +1,57 @@ +package com.baeldung.algorithms.optimization.lp; + +import org.ojalgo.optimisation.ExpressionsBasedModel; +import org.ojalgo.optimisation.Expression; +import org.ojalgo.optimisation.Variable; + +public class OjAlgoAssignmentSolver implements AssignmentSolver { + + public AssignmentSolution solve(double[][] t) { + + int volunteers = t.length; + int locations = t[0].length; + + ExpressionsBasedModel model = new ExpressionsBasedModel(); + Variable[][] x = new Variable[volunteers][locations]; + + // Create binary decision variables + for (int i = 0; i < volunteers; i++) { + for (int j = 0; j < locations; j++) { + x[i][j] = model + .newVariable("Assignment_" + i + "_" + j) + .binary() + .weight(t[i][j]); + } + } + + // Each volunteer is assigned to exactly one location + for (int i = 0; i < volunteers; i++) { + Expression volunteerConstraint = model.addExpression("Volunteer_" + i).level(1); + for (int j = 0; j < locations; j++) { + volunteerConstraint.set(x[i][j], 1); + } + } + + // Each location gets exactly one volunteer + for (int j = 0; j < locations; j++) { + Expression locationConstraint = model.addExpression("Location_" + j).level(1); + for (int i = 0; i < volunteers; i++) { + locationConstraint.set(x[i][j], 1); + } + } + + // Solve + var result = model.minimise(); + double totalCost = result.getValue(); + + // Extract assignment matrix + double[][] assignment = new double[volunteers][locations]; + for (int i = 0; i < volunteers; i++) { + for (int j = 0; j < locations; j++) { + assignment[i][j] = x[i][j].getValue().doubleValue(); + } + } + + return new AssignmentSolution(totalCost, assignment); + } +} diff --git a/azure/src/main/resources/logback.xml b/algorithms-modules/algorithms-optimization/src/main/resources/logback.xml similarity index 100% rename from azure/src/main/resources/logback.xml rename to algorithms-modules/algorithms-optimization/src/main/resources/logback.xml diff --git a/algorithms-modules/algorithms-optimization/src/test/java/com/baeldung/algorithms/optimization/lp/AssignmentSolverTest.java b/algorithms-modules/algorithms-optimization/src/test/java/com/baeldung/algorithms/optimization/lp/AssignmentSolverTest.java new file mode 100644 index 000000000000..df6981b4a82c --- /dev/null +++ b/algorithms-modules/algorithms-optimization/src/test/java/com/baeldung/algorithms/optimization/lp/AssignmentSolverTest.java @@ -0,0 +1,73 @@ +package com.baeldung.algorithms.optimization.lp; + +import java.util.stream.Stream; + +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import static org.assertj.core.api.Assertions.assertThat; + +class AssignmentSolverTest { + + @ParameterizedTest + @MethodSource("assignmentMatrices") + void whenSolveAssignmentMatrixByOjAlgo_thenTheTotalCostIsMinimized(double[][] cost, double expectedTotalCost, double[][] expectedAssignment) { + // given + AssignmentSolver solver = new OjAlgoAssignmentSolver(); + + // when + AssignmentSolution solution = solver.solve(cost); + + // then + assertThat(solution.getTotalCost()).isEqualTo(expectedTotalCost); + assertThat(solution.getAssignment()).isEqualTo(expectedAssignment); + } + + @ParameterizedTest + @MethodSource("assignmentMatrices") + void whenSolveAssignmentMatrixByCommonMaths_thenTheTotalCostIsMinimized(double[][] cost, double expectedTotalCost, double[][] expectedAssignment) { + // given + AssignmentSolver solver = new CommonsMathAssignmentSolver(); + + // when + AssignmentSolution solution = solver.solve(cost); + + // then + assertThat(solution.getTotalCost()).isEqualTo(expectedTotalCost); + assertThat(solution.getAssignment()).isEqualTo(expectedAssignment); + } + + static Stream assignmentMatrices() { + return Stream.of( + Arguments.of( + new double[][] { + {27, 6, 21}, + {18, 12, 9}, + {15, 24, 3} + }, + 27.0, + new double[][] { + {0, 1, 0}, + {1, 0, 0}, + {0, 0, 1} + } + ), + Arguments.of( + new double[][] { + {9, 2, 7, 8}, + {6, 4, 3, 7}, + {5, 8, 1, 8}, + {7, 6, 9, 4} + }, + 13.0, + new double[][] { + {0, 1, 0, 0}, + {1, 0, 0, 0}, + {0, 0, 1, 0}, + {0, 0, 0, 1} + } + ) + ); + } +} diff --git a/algorithms-modules/algorithms-searching/README.md b/algorithms-modules/algorithms-searching/README.md deleted file mode 100644 index 394d14a06cc2..000000000000 --- a/algorithms-modules/algorithms-searching/README.md +++ /dev/null @@ -1,16 +0,0 @@ -## Algorithms - Searching - -This module contains articles about searching algorithms. - -### Relevant articles: - -- [Binary Search Algorithm in Java](https://www.baeldung.com/java-binary-search) -- [Depth First Search in Java](https://www.baeldung.com/java-depth-first-search) -- [Interpolation Search in Java](https://www.baeldung.com/java-interpolation-search) -- [Breadth-First Search Algorithm in Java](https://www.baeldung.com/java-breadth-first-search) -- [String Search Algorithms for Large Texts with Java](https://www.baeldung.com/java-full-text-search-algorithms) -- [Monte Carlo Tree Search for Tic-Tac-Toe Game in Java](https://www.baeldung.com/java-monte-carlo-tree-search) -- [Range Search Algorithm in Java](https://www.baeldung.com/java-range-search) -- [Fast Pattern Matching of Strings Using Suffix Tree in Java](https://www.baeldung.com/java-pattern-matching-suffix-tree) -- [Find the Kth Smallest Element in Two Sorted Arrays in Java](https://www.baeldung.com/java-kth-smallest-element-in-sorted-arrays) -- [Find the First Non-repeating Element of a List](https://www.baeldung.com/java-list-find-first-non-repeating-element) diff --git a/algorithms-modules/algorithms-sorting-2/README.md b/algorithms-modules/algorithms-sorting-2/README.md deleted file mode 100644 index 8e729c48d457..000000000000 --- a/algorithms-modules/algorithms-sorting-2/README.md +++ /dev/null @@ -1,11 +0,0 @@ -### Relevant Articles: - -- [Sorting Strings by Contained Numbers in Java](https://www.baeldung.com/java-sort-strings-contained-numbers) -- [Partitioning and Sorting Arrays with Many Repeated Entries with Java Examples](https://www.baeldung.com/java-sorting-arrays-with-repeated-entries) -- [Selection Sort in Java](https://www.baeldung.com/java-selection-sort) -- [Bubble Sort in Java](https://www.baeldung.com/java-bubble-sort) -- [Insertion Sort in Java](https://www.baeldung.com/java-insertion-sort) -- [Heap Sort in Java](https://www.baeldung.com/java-heap-sort) -- [Counting Sort in Java](https://www.baeldung.com/java-counting-sort) -- [Radix Sort in Java](https://www.baeldung.com/java-radix-sort) -- More articles: [[<-- prev]](/algorithms-modules/algorithms-sorting)[[next -->]](/algorithms-modules/algorithms-sorting-3) diff --git a/algorithms-modules/algorithms-sorting-3/README.md b/algorithms-modules/algorithms-sorting-3/README.md deleted file mode 100644 index 44db9a83a8f5..000000000000 --- a/algorithms-modules/algorithms-sorting-3/README.md +++ /dev/null @@ -1,7 +0,0 @@ -### Relevant Articles: - -- [Bucket Sort in Java](https://www.baeldung.com/java-bucket-sort) -- [Shell Sort in Java](https://www.baeldung.com/java-shell-sort) -- [Gravity/Bead Sort in Java](https://www.baeldung.com/java-gravity-bead-sort) -- [Guide to In-Place Sorting Algorithm Works with a Java Implementation](https://www.baeldung.com/java-in-place-sorting) -- More articles: [[<-- prev]](/algorithms-modules/algorithms-sorting-2) diff --git a/algorithms-modules/algorithms-sorting-3/pom.xml b/algorithms-modules/algorithms-sorting-3/pom.xml index 0440db1c8c61..40db51a5a30c 100644 --- a/algorithms-modules/algorithms-sorting-3/pom.xml +++ b/algorithms-modules/algorithms-sorting-3/pom.xml @@ -1,7 +1,7 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 algorithms-sorting-3 0.0.1-SNAPSHOT @@ -13,7 +13,4 @@ 1.0.0-SNAPSHOT - - - \ No newline at end of file diff --git a/algorithms-modules/algorithms-sorting/README.md b/algorithms-modules/algorithms-sorting/README.md deleted file mode 100644 index 0a9945017f99..000000000000 --- a/algorithms-modules/algorithms-sorting/README.md +++ /dev/null @@ -1,9 +0,0 @@ -## Algorithms - Sorting - -This module contains articles about sorting algorithms. - -### Relevant articles: -- [Merge Sort in Java](https://www.baeldung.com/java-merge-sort) -- [Quicksort Algorithm Implementation in Java](https://www.baeldung.com/java-quicksort) -- [Sorting a String Alphabetically in Java](https://www.baeldung.com/java-sort-string-alphabetically) -- More articles: [[next -->]](/algorithms-modules/algorithms-sorting-2) diff --git a/algorithms-modules/algorithms-sorting/src/main/java/com/baeldung/algorithms/stringsort/AlphanumericSort.java b/algorithms-modules/algorithms-sorting/src/main/java/com/baeldung/algorithms/stringsort/AlphanumericSort.java new file mode 100644 index 000000000000..41e4c0919fd9 --- /dev/null +++ b/algorithms-modules/algorithms-sorting/src/main/java/com/baeldung/algorithms/stringsort/AlphanumericSort.java @@ -0,0 +1,38 @@ +package com.baeldung.algorithms.stringsort; + +import java.util.Arrays; +import java.util.Comparator; + +public class AlphanumericSort { + + public static String lexicographicSort(String input) { + char[] stringChars = input.toCharArray(); + Arrays.sort(stringChars); + return new String(stringChars); + } + + public static String[] naturalAlphanumericSort(String[] arrayToSort) { + Arrays.sort(arrayToSort, new Comparator() { + @Override + public int compare(String s1, String s2) { + return extractInt(s1) - extractInt(s2); + } + + private int extractInt(String str) { + String num = str.replaceAll("\\D+", ""); + return num.isEmpty() ? 0 : Integer.parseInt(num); + } + }); + return arrayToSort; + } + + public static String[] naturalAlphanumericCaseInsensitiveSort(String[] arrayToSort) { + Arrays.sort(arrayToSort, Comparator.comparing((String s) -> s.replaceAll("\\d", "").toLowerCase()) + .thenComparingInt(s -> { + String num = s.replaceAll("\\D+", ""); + return num.isEmpty() ? 0 : Integer.parseInt(num); + }) + .thenComparing(Comparator.naturalOrder())); + return arrayToSort; + } +} \ No newline at end of file diff --git a/algorithms-modules/algorithms-sorting/src/test/java/com/baeldung/algorithms/stringsort/AlphanumericSortUnitTest.java b/algorithms-modules/algorithms-sorting/src/test/java/com/baeldung/algorithms/stringsort/AlphanumericSortUnitTest.java new file mode 100644 index 000000000000..15ca81c4480b --- /dev/null +++ b/algorithms-modules/algorithms-sorting/src/test/java/com/baeldung/algorithms/stringsort/AlphanumericSortUnitTest.java @@ -0,0 +1,31 @@ +package com.baeldung.algorithms.stringsort; + +import org.junit.jupiter.api.Test; + +import java.util.Arrays; + +import static org.assertj.core.api.Assertions.assertThat; + +class AlphanumericSortUnitTest { + + @Test + void givenAlphanumericString_whenLexicographicSort_thenSortLettersFirst() { + String stringToSort = "C4B3A21"; + String sorted = AlphanumericSort.lexicographicSort(stringToSort); + assertThat(sorted).isEqualTo("1234ABC"); + } + + @Test + void givenAlphanumericArrayOfStrings_whenAlphanumericSort_thenSortNaturalOrder() { + String[] arrayToSort = {"file2", "file10", "file0", "file1", "file20"}; + String[] sorted = AlphanumericSort.naturalAlphanumericSort(arrayToSort); + assertThat(Arrays.toString(sorted)).isEqualTo("[file0, file1, file2, file10, file20]"); + } + + @Test + void givenAlphanumericArrayOfStrings_whenAlphanumericCaseInsensitveSort_thenSortNaturalOrder() { + String[] arrayToSort = {"a2", "A10", "b1", "B3", "A2"}; + String[] sorted = AlphanumericSort.naturalAlphanumericCaseInsensitiveSort(arrayToSort); + assertThat(Arrays.toString(sorted)).isEqualTo("[A2, a2, A10, b1, B3]"); + } +} diff --git a/algorithms-modules/pom.xml b/algorithms-modules/pom.xml index 8c84352c254d..751308d56b6f 100644 --- a/algorithms-modules/pom.xml +++ b/algorithms-modules/pom.xml @@ -4,8 +4,8 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 algorithms-modules - algorithms-modules pom + algorithms-modules com.baeldung @@ -26,6 +26,7 @@ algorithms-miscellaneous-9 algorithms-miscellaneous-10 algorithms-numeric + algorithms-optimization algorithms-searching algorithms-sorting algorithms-sorting-2 @@ -37,6 +38,7 @@ 3.6.1 2.7 1.0.1 + 3.3.3
diff --git a/apache-cxf-modules/cxf-aegis/README.md b/apache-cxf-modules/cxf-aegis/README.md deleted file mode 100644 index 1cdb6efbb581..000000000000 --- a/apache-cxf-modules/cxf-aegis/README.md +++ /dev/null @@ -1,3 +0,0 @@ -### Relevant Articles: - -- [Introduction to Apache CXF Aegis Data Binding](https://www.baeldung.com/aegis-data-binding-in-apache-cxf) diff --git a/apache-cxf-modules/cxf-introduction/README.md b/apache-cxf-modules/cxf-introduction/README.md deleted file mode 100644 index 3eef16778513..000000000000 --- a/apache-cxf-modules/cxf-introduction/README.md +++ /dev/null @@ -1,2 +0,0 @@ -### Relevant Articles: -- [Introduction to Apache CXF](https://www.baeldung.com/introduction-to-apache-cxf) diff --git a/apache-cxf-modules/cxf-introduction/pom.xml b/apache-cxf-modules/cxf-introduction/pom.xml index 5e68bf3cbf5f..1aa3a1ea271b 100644 --- a/apache-cxf-modules/cxf-introduction/pom.xml +++ b/apache-cxf-modules/cxf-introduction/pom.xml @@ -47,9 +47,4 @@ - - 4.0.0 - 3.0.0 - - \ No newline at end of file diff --git a/apache-cxf-modules/cxf-jaxrs-implementation/README.md b/apache-cxf-modules/cxf-jaxrs-implementation/README.md deleted file mode 100644 index 28c01e6e3674..000000000000 --- a/apache-cxf-modules/cxf-jaxrs-implementation/README.md +++ /dev/null @@ -1,2 +0,0 @@ -### Relevant Articles: -- [Apache CXF Support for RESTful Web Services](http://www.baeldung.com/apache-cxf-rest-api) diff --git a/apache-cxf-modules/cxf-jaxrs-implementation/pom.xml b/apache-cxf-modules/cxf-jaxrs-implementation/pom.xml index 978c5a51ed5f..3e90179b8a2d 100644 --- a/apache-cxf-modules/cxf-jaxrs-implementation/pom.xml +++ b/apache-cxf-modules/cxf-jaxrs-implementation/pom.xml @@ -31,7 +31,7 @@ jakarta.jws jakarta.jws-api - ${jakarta-jws.version} + ${jakarta.jws.version} jakarta.platform @@ -66,8 +66,6 @@ 4.5.2 - 4.0.0 - 3.0.0 9.0.0 diff --git a/apache-cxf-modules/cxf-spring/README.md b/apache-cxf-modules/cxf-spring/README.md deleted file mode 100644 index c4d55a5c9429..000000000000 --- a/apache-cxf-modules/cxf-spring/README.md +++ /dev/null @@ -1,3 +0,0 @@ -### Relevant Articles: - -- [A Guide to Apache CXF with Spring](https://www.baeldung.com/apache-cxf-with-spring) diff --git a/apache-cxf-modules/pom.xml b/apache-cxf-modules/pom.xml index 9a3721f7b7f9..d88e41939e7b 100644 --- a/apache-cxf-modules/pom.xml +++ b/apache-cxf-modules/pom.xml @@ -8,7 +8,6 @@ pom apache-cxf-modules - com.baeldung parent-modules @@ -38,6 +37,8 @@ 4.0.0 3.1.0 + 3.0.0 + 4.0.0 \ No newline at end of file diff --git a/apache-cxf-modules/sse-jaxrs/README.md b/apache-cxf-modules/sse-jaxrs/README.md deleted file mode 100644 index ee85940b8ad6..000000000000 --- a/apache-cxf-modules/sse-jaxrs/README.md +++ /dev/null @@ -1,3 +0,0 @@ -### Relevant Articles: - -- [Server-Sent Events (SSE) in JAX-RS](https://www.baeldung.com/java-ee-jax-rs-sse) diff --git a/apache-httpclient-2/README.md b/apache-httpclient-2/README.md deleted file mode 100644 index e6060ba7beeb..000000000000 --- a/apache-httpclient-2/README.md +++ /dev/null @@ -1,20 +0,0 @@ -## Apache HttpClient - -This module contains articles about Apache HttpClient - -### The Course - -The "REST With Spring" Classes: http://bit.ly/restwithspring - -### Relevant Articles: - -- [How to Set TLS Version in Apache HttpClient](https://www.baeldung.com/apache-httpclient-tls) -- [How To Get Cookies From the Apache HttpClient Response](https://www.baeldung.com/java-apache-httpclient-cookies) -- [Advanced Apache HttpClient Configuration](https://www.baeldung.com/httpclient-advanced-config) -- [Apache HttpClient – Follow Redirects for POST](https://www.baeldung.com/httpclient-redirect-on-http-post) -- [Apache HttpAsyncClient Tutorial](https://www.baeldung.com/httpasyncclient-tutorial) -- [Custom User-Agent in Apache HttpClient](https://www.baeldung.com/httpclient-user-agent-header) -- [Apache HttpClient – Do Not Follow Redirects](https://www.baeldung.com/httpclient-stop-follow-redirect) -- [Apache HttpClient – Cancel Request](https://www.baeldung.com/httpclient-cancel-request) - -- More articles: [[<-- prev]](../apache-httpclient) diff --git a/apache-httpclient-2/pom.xml b/apache-httpclient-2/pom.xml index bd9f26a42d38..3312a907a560 100644 --- a/apache-httpclient-2/pom.xml +++ b/apache-httpclient-2/pom.xml @@ -64,6 +64,16 @@ 11 11 + 11 + + + + org.apache.maven.plugins + maven-surefire-plugin + + + ERROR + diff --git a/apache-httpclient/README.md b/apache-httpclient/README.md deleted file mode 100644 index 455c26c5244d..000000000000 --- a/apache-httpclient/README.md +++ /dev/null @@ -1,16 +0,0 @@ -## Apache HttpClient - -This module contains articles about Apache HttpClient - -### The Course - -The "REST With Spring" Classes: http://bit.ly/restwithspring - -### Relevant Articles: -- [Apache HttpClient Cookbook](https://www.baeldung.com/apache-httpclient-cookbook) -- [Multipart Upload with Apache HttpClient](https://www.baeldung.com/httpclient-multipart-upload) -- [Apache HttpClient Connection Management](https://www.baeldung.com/httpclient-connection-management) -- [Reading an HTTP Response Body as a String in Java](https://www.baeldung.com/java-http-response-body-as-string) -- [Apache HttpClient vs. CloseableHttpClient](https://www.baeldung.com/apache-httpclient-vs-closeablehttpclient) -- [Enabling Logging for Apache HttpClient](https://www.baeldung.com/apache-httpclient-enable-logging) -- More articles: [[next -->]](../apache-httpclient-2) diff --git a/apache-httpclient/pom.xml b/apache-httpclient/pom.xml index 445cf14ca46a..cc8893190ec7 100644 --- a/apache-httpclient/pom.xml +++ b/apache-httpclient/pom.xml @@ -80,6 +80,17 @@ 11 11 + 11 + + + + org.apache.maven.plugins + maven-surefire-plugin + ${maven-surefire-plugin.version} + + + ERROR + @@ -92,7 +103,7 @@ - 5.6.1 + 5.11.2 3.9.1 5.2.5 diff --git a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpClientUnitTest.java b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpClientUnitTest.java index 89142793e5ba..2d28545b78fa 100644 --- a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpClientUnitTest.java +++ b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpClientUnitTest.java @@ -2,6 +2,8 @@ import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.IOException; import java.net.URI; @@ -10,6 +12,7 @@ import java.net.http.HttpResponse; class HttpClientUnitTest { + private final Logger logger = LoggerFactory.getLogger(getClass()); public static final String DUMMY_URL = "https://postman-echo.com/get"; @Test @@ -19,12 +22,12 @@ void whenUseHttpClient_thenCorrect() throws IOException, InterruptedException { // synchronous response HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofString()); - System.out.println(response.body()); + logger.debug(response.body()); // asynchronous response client.sendAsync(request, HttpResponse.BodyHandlers.ofString()) .thenApply(HttpResponse::body) - .thenAccept(System.out::println) + .thenAccept(logger::debug) .join(); } } diff --git a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpUrlConnectionUnitTest.java b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpUrlConnectionUnitTest.java index eafc25206d5a..517dbf6e2e1d 100644 --- a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpUrlConnectionUnitTest.java +++ b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/HttpUrlConnectionUnitTest.java @@ -2,6 +2,8 @@ import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.BufferedReader; import java.io.IOException; @@ -14,7 +16,7 @@ import static org.junit.jupiter.api.Assertions.assertNotNull; public class HttpUrlConnectionUnitTest { - + private final Logger logger = LoggerFactory.getLogger(getClass()); public static final String DUMMY_URL = "https://postman-echo.com/get"; @Test @@ -32,6 +34,6 @@ void whenUseHttpUrlConnection_thenCorrect() throws IOException, URISyntaxExcepti in.close(); assertNotNull(response.toString()); - System.out.println("Response -> " + response.toString()); + logger.debug("Response -> {}", response.toString()); } } diff --git a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringRestTemplateUnitTest.java b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringRestTemplateUnitTest.java index e06cc165f09f..aac0860ea49d 100644 --- a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringRestTemplateUnitTest.java +++ b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringRestTemplateUnitTest.java @@ -1,17 +1,19 @@ package com.baeldung.httpclient.readresponsebodystring; import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.springframework.web.client.RestTemplate; public class SpringRestTemplateUnitTest { - + private final Logger logger = LoggerFactory.getLogger(getClass()); public static final String DUMMY_URL = "https://postman-echo.com/get"; @Test public void whenUseRestTemplate_thenCorrect() { RestTemplate restTemplate = new RestTemplate(); String response = restTemplate.getForObject(DUMMY_URL, String.class); - System.out.println(response); + logger.debug(response); } } diff --git a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringWebClientUnitTest.java b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringWebClientUnitTest.java index df71bab98357..6d96944316ed 100644 --- a/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringWebClientUnitTest.java +++ b/apache-httpclient/src/test/java/com/baeldung/httpclient/readresponsebodystring/SpringWebClientUnitTest.java @@ -1,10 +1,13 @@ package com.baeldung.httpclient.readresponsebodystring; import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.springframework.web.reactive.function.client.WebClient; import reactor.core.publisher.Mono; public class SpringWebClientUnitTest { + private final Logger logger = LoggerFactory.getLogger(getClass()); public static final String DUMMY_URL = "https://postman-echo.com/get"; @Test @@ -12,6 +15,6 @@ void whenUseWebClientRetrieve_thenCorrect() { WebClient webClient = WebClient.create(DUMMY_URL); Mono body = webClient.get().retrieve().bodyToMono(String.class); String s = body.block(); - System.out.println(s); + logger.debug(s); } } diff --git a/apache-httpclient4/README.md b/apache-httpclient4/README.md index 0bc4ac8e83d6..84835826c6ad 100644 --- a/apache-httpclient4/README.md +++ b/apache-httpclient4/README.md @@ -2,17 +2,6 @@ This module contains articles about Apache HttpClient 4.5 -### Relevant Articles - -- [Apache HttpClient – Cancel Request](https://www.baeldung.com/httpclient-cancel-request) -- [Apache HttpClient with SSL](https://www.baeldung.com/httpclient-ssl) -- [Apache HttpClient Timeout](https://www.baeldung.com/httpclient-timeout) -- [Custom HTTP Header with the Apache HttpClient](https://www.baeldung.com/httpclient-custom-http-header) -- [Apache HttpClient vs. CloseableHttpClient](https://www.baeldung.com/apache-httpclient-vs-closeablehttpclient) -- [Expand Shortened URLs with Apache HttpClient](https://www.baeldung.com/apache-httpclient-expand-url) -- [Retrying Requests using Apache HttpClient](https://www.baeldung.com/java-retrying-requests-using-apache-httpclient) -- [Apache HttpClient – Follow Redirects for POST](https://www.baeldung.com/httpclient-redirect-on-http-post) - ### Running the Tests To run the live tests, use the command: mvn clean install -Plive This will start an embedded Jetty server on port 8082 using the Cargo plugin configured in the pom.xml file, diff --git a/apache-httpclient4/pom.xml b/apache-httpclient4/pom.xml index c218629940c0..b78dcb99b03d 100644 --- a/apache-httpclient4/pom.xml +++ b/apache-httpclient4/pom.xml @@ -100,7 +100,6 @@ - org.apache.commons @@ -192,6 +191,17 @@ true + + + org.apache.maven.plugins + maven-surefire-plugin + + + ERROR + + + + diff --git a/apache-httpclient4/src/test/resources/mockserver.properties b/apache-httpclient4/src/test/resources/mockserver.properties new file mode 100644 index 000000000000..23dc75415cbc --- /dev/null +++ b/apache-httpclient4/src/test/resources/mockserver.properties @@ -0,0 +1 @@ +mockserver.logLevel=OFF \ No newline at end of file diff --git a/apache-kafka-2/README.md b/apache-kafka-2/README.md index 9f244db4616f..f43d51c20cd7 100644 --- a/apache-kafka-2/README.md +++ b/apache-kafka-2/README.md @@ -4,15 +4,3 @@ This module contains articles about Apache Kafka. ##### Building the project You can build the project from the command line using: *mvn clean install*, or in an IDE. - -### Relevant Articles: -- [Guide to Check if Apache Kafka Server Is Running](https://www.baeldung.com/apache-kafka-check-server-is-running) -- [Introduction to Apache Kafka](https://www.baeldung.com/apache-kafka) -- [Read Multiple Messages with Apache Kafka](https://www.baeldung.com/kafka-read-multiple-messages) -- [Creating a Kafka Listener Using the Consumer API](https://www.baeldung.com/kafka-create-listener-consumer-api) -- [Introduction to KafkaStreams in Java](https://www.baeldung.com/java-kafka-streams) -- [Building a Data Pipeline with Flink and Kafka](https://www.baeldung.com/kafka-flink-data-pipeline) -- [Kafka Topic Creation Using Java](https://www.baeldung.com/kafka-topic-creation) -- [Using Kafka MockProducer](https://www.baeldung.com/kafka-mockproducer) -- [Using Kafka MockConsumer](https://www.baeldung.com/kafka-mockconsumer) -- [Kafka Connect Example with MQTT and MongoDB](https://www.baeldung.com/kafka-connect-mqtt-mongodb) \ No newline at end of file diff --git a/apache-kafka-2/pom.xml b/apache-kafka-2/pom.xml index ebb4d1a373d7..d3ad68f5c97f 100644 --- a/apache-kafka-2/pom.xml +++ b/apache-kafka-2/pom.xml @@ -119,9 +119,23 @@ + + + + org.apache.maven.plugins + maven-surefire-plugin + + + src/test/resources/logback.xml + + + + + + 5.7.0 - 3.6.1 + 3.9.0 1.19.3 1.19.3 2.15.2 @@ -130,4 +144,4 @@ 3.0.0 - \ No newline at end of file + diff --git a/apache-kafka-2/src/test/java/com/baeldung/kafkastreams/KafkaStreamsLiveTest.java b/apache-kafka-2/src/test/java/com/baeldung/kafkastreams/KafkaStreamsLiveTest.java index 09770148e084..56b070bb78e7 100644 --- a/apache-kafka-2/src/test/java/com/baeldung/kafkastreams/KafkaStreamsLiveTest.java +++ b/apache-kafka-2/src/test/java/com/baeldung/kafkastreams/KafkaStreamsLiveTest.java @@ -40,7 +40,7 @@ public void shouldTestKafkaStreams() throws InterruptedException { .getName()); streamsConfiguration.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 1000); streamsConfiguration.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); - + // Use a temporary directory for storing state, which will be automatically removed after the test. try { Path stateDirectory = Files.createTempDirectory("kafka-streams"); @@ -75,4 +75,4 @@ public void shouldTestKafkaStreams() throws InterruptedException { Thread.sleep(30000); streams.close(); } -} +} \ No newline at end of file diff --git a/apache-kafka-2/src/test/resources/logback.xml b/apache-kafka-2/src/test/resources/logback.xml index 6156c2188e68..cd80b11d583d 100644 --- a/apache-kafka-2/src/test/resources/logback.xml +++ b/apache-kafka-2/src/test/resources/logback.xml @@ -4,6 +4,8 @@ %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n + + diff --git a/apache-kafka-3/README.md b/apache-kafka-3/README.md index 7ff6f4e5f70b..8e8a560259ba 100644 --- a/apache-kafka-3/README.md +++ b/apache-kafka-3/README.md @@ -5,9 +5,3 @@ This module contains articles about Apache Kafka. ##### Building the project You can build the project from the command line using: *mvn clean install*, or in an IDE. -### Relevant Articles: -- [Get Last N Messages in Apache Kafka Topic](https://www.baeldung.com/java-apache-kafka-get-last-n-messages) -- [Get Partition Count for a Topic in Kafka](https://www.baeldung.com/java-kafka-partition-count-topic) -- [Retries With Kafka Producer](https://www.baeldung.com/kafka-producer-retries) -- [Handling Kafka Producer TimeOutException with Java](https://www.baeldung.com/java-kafka-timeoutexception) - diff --git a/apache-kafka-3/docker-compose.yml b/apache-kafka-3/docker-compose.yml new file mode 100644 index 000000000000..b68763bbeec4 --- /dev/null +++ b/apache-kafka-3/docker-compose.yml @@ -0,0 +1,41 @@ +version: '3.8' +services: + zookeeper: + image: confluentinc/cp-zookeeper:7.4.0 + hostname: zookeeper + container_name: zookeeper + ports: + - "2181:2181" + environment: + ZOOKEEPER_CLIENT_PORT: 2181 + ZOOKEEPER_TICK_TIME: 2000 + + kafka: + image: confluentinc/cp-kafka:7.4.0 + hostname: kafka + container_name: kafka + depends_on: + - zookeeper + ports: + - "9092:9092" + environment: + KAFKA_BROKER_ID: 1 + KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092 + KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 + KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 + KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true' + KAFKA_NUM_PARTITIONS: 6 + + kafka-ui: + image: provectuslabs/kafka-ui:latest + container_name: kafka-ui + depends_on: + - kafka + ports: + - "8080:8080" + environment: + KAFKA_CLUSTERS_0_NAME: local + KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:29092 \ No newline at end of file diff --git a/apache-kafka-3/pom.xml b/apache-kafka-3/pom.xml index 4c28fb9d097b..b58911b1b438 100644 --- a/apache-kafka-3/pom.xml +++ b/apache-kafka-3/pom.xml @@ -1,7 +1,7 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 apache-kafka-3 apache-kafka-3 @@ -49,7 +49,7 @@ - 3.8.0 + 3.9.1 2.15.2 1.19.3 1.19.3 diff --git a/apache-kafka-3/src/main/java/com/baeldung/kafka/KafkaAddPartitionExample.java b/apache-kafka-3/src/main/java/com/baeldung/kafka/KafkaAddPartitionExample.java new file mode 100644 index 000000000000..b0958be70f1f --- /dev/null +++ b/apache-kafka-3/src/main/java/com/baeldung/kafka/KafkaAddPartitionExample.java @@ -0,0 +1,23 @@ +package com.baeldung.kafka; + +import java.util.Collections; +import java.util.Properties; + +import org.apache.kafka.clients.admin.AdminClient; +import org.apache.kafka.clients.admin.NewPartitions; + +public class KafkaAddPartitionExample { + + public static void main(String[] args) { + Properties props = new Properties(); + props.put("bootstrap.servers", "localhost:9092"); + try (AdminClient adminClient = AdminClient.create(props)) { + adminClient.createPartitions(Collections.singletonMap("my-topic", NewPartitions.increaseTo(3))) + .all() + .get(); + } catch (Exception e) { + throw new RuntimeException(e); + } + } + +} diff --git a/apache-kafka-3/src/main/java/com/baeldung/kafka/partitions/KafkaMultiplePartitionsDemo.java b/apache-kafka-3/src/main/java/com/baeldung/kafka/partitions/KafkaMultiplePartitionsDemo.java new file mode 100644 index 000000000000..50d079942adc --- /dev/null +++ b/apache-kafka-3/src/main/java/com/baeldung/kafka/partitions/KafkaMultiplePartitionsDemo.java @@ -0,0 +1,263 @@ +package com.baeldung.kafka.partitions; + +import java.time.Duration; +import java.util.Arrays; +import java.util.Collection; +import java.util.HashMap; +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; + +import org.apache.kafka.clients.consumer.ConsumerRebalanceListener; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.clients.producer.KafkaProducer; +import org.apache.kafka.clients.producer.ProducerConfig; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class KafkaMultiplePartitionsDemo { + + private static final Logger logger = LoggerFactory.getLogger(KafkaMultiplePartitionsDemo.class); + private final KafkaProducer producer; + private final String bootstrapServers; + + public KafkaMultiplePartitionsDemo(String bootstrapServers) { + this.bootstrapServers = bootstrapServers; + this.producer = createProducer(); + } + + private KafkaProducer createProducer() { + Properties props = new Properties(); + props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers); + props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); + props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); + props.put(ProducerConfig.ACKS_CONFIG, "all"); + return new KafkaProducer<>(props); + } + + public void sendMessagesWithKey() { + String key = "user-123"; + + for (int i = 0; i < 5; i++) { + ProducerRecord record = new ProducerRecord<>("user-events", key, "Event " + i); + + producer.send(record, (metadata, exception) -> { + if (exception == null) { + logger.info("Key: {}, Partition: {}, Offset: {}", key, metadata.partition(), metadata.offset()); + } + }); + } + producer.flush(); + } + + public Map sendMessagesWithoutKey() { + Map partitionCounts = new HashMap<>(); + + for (int i = 0; i < 100; i++) { + ProducerRecord record = new ProducerRecord<>("events", null, // no key + "Message " + i); + + producer.send(record, (metadata, exception) -> { + if (exception == null) { + synchronized (partitionCounts) { + partitionCounts.merge(metadata.partition(), 1, Integer::sum); + } + } + }); + } + producer.flush(); + logger.info("Distribution across partitions: {}", partitionCounts); + return partitionCounts; + } + + public void demonstratePartitionOrdering() throws InterruptedException { + String orderId = "order-789"; + String[] events = { "created", "validated", "paid", "shipped", "delivered" }; + + for (String event : events) { + ProducerRecord record = new ProducerRecord<>("orders", orderId, event); + + producer.send(record, (metadata, exception) -> { + if (exception == null) { + logger.info("Event: {} -> Partition: {}, Offset: {}", event, metadata.partition(), metadata.offset()); + } + }); + // small delay to demonstrate sequential processing + Thread.sleep(100); + } + producer.flush(); + } + + public void demonstrateCrossPartitionBehavior() { + long startTime = System.currentTimeMillis(); + + // these will likely go to different partitions + producer.send(new ProducerRecord<>("events", "key-A", "First at " + (System.currentTimeMillis() - startTime) + "ms")); + producer.send(new ProducerRecord<>("events", "key-B", "Second at " + (System.currentTimeMillis() - startTime) + "ms")); + producer.send(new ProducerRecord<>("events", "key-C", "Third at " + (System.currentTimeMillis() - startTime) + "ms")); + + producer.flush(); + } + + public void close() { + if (producer != null) { + producer.close(); + } + } + + public void createConsumerGroup() { + Properties props = new Properties(); + props.put("bootstrap.servers", bootstrapServers); + props.put("group.id", "order-processors"); + props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); + props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); + props.put("auto.offset.reset", "earliest"); + + KafkaConsumer consumer = new KafkaConsumer<>(props); + consumer.subscribe(Arrays.asList("orders")); + + int recordCount = 0; + while (recordCount < 10) { // process limited records for demo + ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); + + for (ConsumerRecord record : records) { + logger.info("Consumer: {}, Partition: {}, Offset: {}, Value: {}", Thread.currentThread() + .getName(), record.partition(), record.offset(), record.value()); + recordCount++; + } + consumer.commitSync(); + } + consumer.close(); + } + + public void startMultipleGroups() { + String[] groupIds = { "analytics-group", "audit-group", "notification-group" }; + CountDownLatch latch = new CountDownLatch(groupIds.length); + for (String groupId : groupIds) { + startConsumerGroup(groupId, latch); + } + + try { + latch.await(10, TimeUnit.SECONDS); + } catch (InterruptedException e) { + Thread.currentThread() + .interrupt(); + } + } + + private void startConsumerGroup(String groupId, CountDownLatch latch) { + Properties props = new Properties(); + props.put("bootstrap.servers", bootstrapServers); + props.put("group.id", groupId); + props.put("auto.offset.reset", "earliest"); + props.put("key.deserializer", StringDeserializer.class.getName()); + props.put("value.deserializer", StringDeserializer.class.getName()); + + new Thread(() -> { + try (KafkaConsumer consumer = new KafkaConsumer<>(props)) { + consumer.subscribe(Arrays.asList("orders")); + + int recordCount = 0; + while (recordCount < 5) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); + recordCount += processRecordsForGroup(groupId, records); + } + } finally { + latch.countDown(); + } + }).start(); + } + + private int processRecordsForGroup(String groupId, ConsumerRecords records) { + int count = 0; + for (ConsumerRecord record : records) { + logger.info("[{}] Processing: {}", groupId, record.value()); + count++; + } + return count; + } + + public void configureCooperativeRebalancing() { + Properties props = new Properties(); + props.put("bootstrap.servers", bootstrapServers); + props.put("group.id", "cooperative-group"); + props.put("partition.assignment.strategy", "org.apache.kafka.clients.consumer.CooperativeStickyAssignor"); + props.put("key.deserializer", StringDeserializer.class.getName()); + props.put("value.deserializer", StringDeserializer.class.getName()); + props.put("auto.offset.reset", "earliest"); + + KafkaConsumer consumer = new KafkaConsumer<>(props); + + consumer.subscribe(Arrays.asList("orders"), new ConsumerRebalanceListener() { + @Override + public void onPartitionsRevoked(Collection partitions) { + logger.info("Revoked partitions: {}", partitions); + // complete processing of current records + } + + @Override + public void onPartitionsAssigned(Collection partitions) { + logger.info("Assigned partitions: {}", partitions); + // initialize any partition-specific state + } + }); + + // process a few records to demonstrate + int recordCount = 0; + while (recordCount < 5) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); + recordCount += records.count(); + } + + consumer.close(); + } + + public void processWithManualCommit() { + Properties props = new Properties(); + props.put("bootstrap.servers", bootstrapServers); + props.put("group.id", "manual-commit-group"); + props.put("enable.auto.commit", "false"); + props.put("max.poll.records", "10"); + props.put("key.deserializer", StringDeserializer.class.getName()); + props.put("value.deserializer", StringDeserializer.class.getName()); + props.put("auto.offset.reset", "earliest"); + + KafkaConsumer consumer = new KafkaConsumer<>(props); + consumer.subscribe(Arrays.asList("orders")); + + int totalProcessed = 0; + while (totalProcessed < 10) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); + + for (ConsumerRecord record : records) { + try { + processOrder(record); + totalProcessed++; + } catch (Exception e) { + logger.error("Processing failed for offset: {}", record.offset(), e); + break; + } + } + + if (!records.isEmpty()) { + consumer.commitSync(); + logger.info("Committed {} records", records.count()); + } + } + + consumer.close(); + } + + private void processOrder(ConsumerRecord record) { + // simulate order processing + logger.info("Processing order: {}", record.value()); + // this section is mostly your part of implementation, which is out of bounds of the article topic coverage + } +} \ No newline at end of file diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/avro-sink-config.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/avro-sink-config.json new file mode 100644 index 000000000000..90277f6d5bd2 --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/avro-sink-config.json @@ -0,0 +1,14 @@ +{ + "name": "avro-elasticsearch-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "avro_logs", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "true", + "schema.ignore": "false", + "value.converter": "io.confluent.connect.avro.AvroConverter", + "value.converter.schema.registry.url": "http://schema-registry:8081" + } + } + \ No newline at end of file diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/dlq-sink-config.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/dlq-sink-config.json new file mode 100644 index 000000000000..a473c16e5421 --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/dlq-sink-config.json @@ -0,0 +1,20 @@ +{ + "name": "dlq-elasticsearch-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "failing-topic", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "true", + "schema.ignore": "true", + + "behavior.on.malformed.documents": "ignore", + "errors.tolerance": "all", + "errors.log.enable": "true", + "errors.log.include.messages": "true", + + "errors.deadletterqueue.topic.name": "dlq-logs", + "errors.deadletterqueue.topic.replication.factor": "1", + "errors.deadletterqueue.context.headers.enable": "true" + } +} diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/error-handling-sink-config.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/error-handling-sink-config.json new file mode 100644 index 000000000000..ace275d8d4de --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/error-handling-sink-config.json @@ -0,0 +1,16 @@ +{ + "name": "error-handling-elasticsearch-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "test-error-handling", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "true", + "schema.ignore": "true", + "behavior.on.malformed.documents": "warn", + "behavior.on.error": "LOG", + "errors.tolerance": "all", + "errors.log.enable": "true", + "errors.log.include.messages": "true" + } +} \ No newline at end of file diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/flush-optimization-sink.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/flush-optimization-sink.json new file mode 100644 index 000000000000..bf33f926e495 --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/flush-optimization-sink.json @@ -0,0 +1,23 @@ +{ + "name": "flush-optimization-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "logs", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "true", + "schema.ignore": "true", + + "flush.size": "500", + "linger.ms": "1000", + "batch.size": "200", + "max.buffered.records": "10000", + "max.retries": "10", + "retry.backoff.ms": "500", + "max.in.flight.requests": "10", + "behavior.on.malformed.documents": "warn", + "write.method": "bulk", + "request.timeout.ms": "30000", + "connection.timeout.ms": "10000" + } +} diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/message-id-sink-config.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/message-id-sink-config.json new file mode 100644 index 000000000000..9f92e00a96bd --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/message-id-sink-config.json @@ -0,0 +1,19 @@ +{ + "name": "message-id-elasticsearch-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "message_id_logs", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "false", + "schema.ignore": "true", + "value.converter": "org.apache.kafka.connect.json.JsonConverter", + "value.converter.schemas.enable": "false", + "transforms": "InsertKey,ExtractId", + "transforms.InsertKey.type": "org.apache.kafka.connect.transforms.ValueToKey", + "transforms.InsertKey.fields": "user_id", + "transforms.ExtractId.type": "org.apache.kafka.connect.transforms.ExtractField$Key", + "transforms.ExtractId.field": "user_id" + } +} + \ No newline at end of file diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/partition-key-sink-config.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/partition-key-sink-config.json new file mode 100644 index 000000000000..4bb4a4b30d34 --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/partition-key-sink-config.json @@ -0,0 +1,12 @@ +{ + "name": "partition-key-elasticsearch-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "partition_key_logs", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "false", + "schema.ignore": "true", + "key.converter": "org.apache.kafka.connect.storage.StringConverter" + } +} diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/test-connector.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/test-connector.json new file mode 100644 index 000000000000..611c44e25be1 --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/test-connector.json @@ -0,0 +1,12 @@ +{ + "name": "elasticsearch-sink-connector-test", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "logs", + "connection.url": "http://elasticsearch:9200", + "type.name": "_doc", + "key.ignore": "true", + "schema.ignore": "true" + } +} \ No newline at end of file diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/timestamp-index-sink.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/timestamp-index-sink.json new file mode 100644 index 000000000000..f6ac3bc5a75b --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/timestamp-index-sink.json @@ -0,0 +1,20 @@ +{ + "name": "timestamp-index-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "test", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "true", + "schema.ignore": "true", + "transforms": "TimestampConverter, TimestampRouter", + "transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value", + "transforms.TimestampConverter.field": "timestamp", + "transforms.TimestampConverter.target.type": "string", + "transforms.TimestampConverter.format": "yyyy-MM-dd'T'HH:mm:ssX", + "transforms.TimestampRouter.type": "org.apache.kafka.connect.transforms.TimestampRouter", + "transforms.TimestampRouter.topic.format": "${topic}-${timestamp}", + "transforms.TimestampRouter.timestamp.format": "yyyy-MM-dd", + "flush.synchronously": "true" + } +} \ No newline at end of file diff --git a/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/timestamp-transform-sink.json b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/timestamp-transform-sink.json new file mode 100644 index 000000000000..38aa7bf3a0e6 --- /dev/null +++ b/apache-kafka-3/src/main/resources/kafka-connector-elasticsearch/timestamp-transform-sink.json @@ -0,0 +1,17 @@ +{ + "name": "timestamp-transform-sink", + "config": { + "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector", + "tasks.max": "1", + "topics": "epoch_logs", + "connection.url": "http://elasticsearch:9200", + "key.ignore": "true", + "schema.ignore": "true", + "transforms": "TimestampConverter", + "transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value", + "transforms.TimestampConverter.field": "timestamp", + "transforms.TimestampConverter.target.type": "string", + "transforms.TimestampConverter.format": "yyyy-MM-dd'T'HH:mm:ssZ" + } +} + \ No newline at end of file diff --git a/apache-kafka-4/README.md b/apache-kafka-4/README.md new file mode 100644 index 000000000000..f43d51c20cd7 --- /dev/null +++ b/apache-kafka-4/README.md @@ -0,0 +1,6 @@ +## Apache Kafka + +This module contains articles about Apache Kafka. + +##### Building the project +You can build the project from the command line using: *mvn clean install*, or in an IDE. diff --git a/apache-kafka-4/pom.xml b/apache-kafka-4/pom.xml new file mode 100644 index 000000000000..1cad893182ff --- /dev/null +++ b/apache-kafka-4/pom.xml @@ -0,0 +1,111 @@ + + + 4.0.0 + apache-kafka-4 + apache-kafka-4 + + + com.baeldung + parent-modules + 1.0.0-SNAPSHOT + + + + + org.apache.kafka + kafka-clients + ${kafka.version} + + + org.apache.kafka + kafka-streams + ${kafka.version} + + + org.slf4j + slf4j-api + ${org.slf4j.version} + + + com.fasterxml.jackson.core + jackson-databind + ${jackson.databind.version} + + + org.openjdk.jmh + jmh-core + ${jmh-core.version} + + + org.openjdk.jmh + jmh-generator-annprocess + ${jmh-generator.version} + + + org.testcontainers + kafka + ${testcontainers-kafka.version} + test + + + org.testcontainers + junit-jupiter + ${testcontainers-jupiter.version} + test + + + org.awaitility + awaitility + ${awaitility.version} + test + + + org.awaitility + awaitility-proxy + ${awaitility.version} + test + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + + src/test/resources/logback.xml + + + + + org.apache.maven.plugins + maven-compiler-plugin + ${maven-compiler-plugin.version} + + ${java.version} + ${java.version} + + + org.openjdk.jmh + jmh-generator-annprocess + ${jmh-generator.version} + + + + + + + + + 21 + 3.9.0 + 1.19.3 + 1.19.3 + 2.15.2 + 3.0.0 + + + diff --git a/apache-kafka-4/src/main/java/com/baeldung/idempotentproducer/benchmark/BenchmarkRunner.java b/apache-kafka-4/src/main/java/com/baeldung/idempotentproducer/benchmark/BenchmarkRunner.java new file mode 100644 index 000000000000..c7a5ebd134d6 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/idempotentproducer/benchmark/BenchmarkRunner.java @@ -0,0 +1,18 @@ +package com.baeldung.idempotentproducer.benchmark; + +import org.openjdk.jmh.runner.Runner; +import org.openjdk.jmh.runner.RunnerException; +import org.openjdk.jmh.runner.options.Options; +import org.openjdk.jmh.runner.options.OptionsBuilder; + +public class BenchmarkRunner { + + public static void main(String[] args) throws RunnerException { + Options opt = new OptionsBuilder() + .include(IdempotenceBenchmark.class.getSimpleName()) + .build(); + + new Runner(opt).run(); + } + +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/idempotentproducer/benchmark/IdempotenceBenchmark.java b/apache-kafka-4/src/main/java/com/baeldung/idempotentproducer/benchmark/IdempotenceBenchmark.java new file mode 100644 index 000000000000..4064d3e9ff48 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/idempotentproducer/benchmark/IdempotenceBenchmark.java @@ -0,0 +1,107 @@ +package com.baeldung.idempotentproducer.benchmark; + +import org.apache.kafka.clients.admin.*; +import org.apache.kafka.clients.producer.*; +import org.apache.kafka.common.serialization.LongSerializer; +import org.apache.kafka.common.serialization.ByteArraySerializer; +import org.openjdk.jmh.annotations.*; + +import java.util.List; +import java.util.Properties; +import java.util.concurrent.Future; + +@BenchmarkMode(Mode.Throughput) +@Warmup(iterations = 2, time = 5) +@Measurement(iterations = 5, time = 20) +@Fork(2) +@State(Scope.Benchmark) +public class IdempotenceBenchmark { + + private static final String BOOTSTRAP = "localhost:29092,localhost:39092,localhost:49092"; + private static final int MESSAGES = 30000; + private static final String TOPIC = "benchmark-topic"; + private static final int PARTITIONS = 6; + private static final short REPLICATION_FACTOR = 3; + + @Param({ "true", "false" }) + public boolean idempotent; + + private final byte[] value = new byte[1024]; + + private Producer producer; + private long counter = 0; + + @Setup(Level.Trial) + public void setupTrial() throws Exception { + counter = 0; + createTopic(); + + producer = new KafkaProducer<>(props(idempotent)); + + // ensure topic is created + producer.partitionsFor(TOPIC); + for (int p = 0; p < PARTITIONS; p++) { + producer.send(new ProducerRecord<>(TOPIC, p, -1L, value)).get(); + } + } + + @TearDown(Level.Trial) + public void shutdownTrial() { + if (producer != null) { + producer.close(); + } + } + + @Benchmark + @OperationsPerInvocation(MESSAGES) + public void sendMessages() throws Exception { + Future[] futures = new Future[MESSAGES]; + for (int i = 0; i < MESSAGES; i++) { + long key = counter++; + futures[i] = producer.send(new ProducerRecord<>(TOPIC, key, value)); + } + + for (Future f : futures) { + f.get(); + } + } + + private static Properties props(boolean idempotent) { + Properties props = new Properties(); + props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP); + props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, LongSerializer.class.getName()); + props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class.getName()); + + props.put(ProducerConfig.ACKS_CONFIG, "all"); + props.put(ProducerConfig.RETRIES_CONFIG, Integer.toString(Integer.MAX_VALUE)); + props.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, "5"); + + props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, String.valueOf(idempotent)); + + props.put(ProducerConfig.LINGER_MS_CONFIG, "5"); + props.put(ProducerConfig.BATCH_SIZE_CONFIG, Integer.toString(32 * 1024)); + props.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "none"); + props.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, "30000"); + props.put(ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG, "120000"); + + return props; + } + + private static void createTopic() throws Exception { + Properties props = new Properties(); + props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP); + + try (AdminClient admin = AdminClient.create(props)) { + boolean exists = admin.listTopics() + .names() + .get() + .contains(TOPIC); + + if (!exists) { + admin.createTopics(List.of(new NewTopic(TOPIC, PARTITIONS, REPLICATION_FACTOR))) + .all() + .get(); + } + } + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/assignsubscribe/SubscriberUsingAssign.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/assignsubscribe/SubscriberUsingAssign.java new file mode 100644 index 000000000000..4d2e9859e62f --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/assignsubscribe/SubscriberUsingAssign.java @@ -0,0 +1,48 @@ +package com.baeldung.kafka.assignsubscribe; + +import java.time.Duration; +import java.util.Arrays; +import java.util.Properties; + +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class SubscriberUsingAssign { + + public static void main(String[] args) { + Logger logger = LoggerFactory.getLogger(SubscriberUsingAssign.class); + + // Create and Set Consumer Properties + Properties properties = new Properties(); + String bootstrapServer = "127.0.0.1:9092"; + String groupId = "baeldung-consumer-group"; + properties.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServer); + properties.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + properties.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + properties.setProperty(ConsumerConfig.GROUP_ID_CONFIG, groupId); + + // Create Kafka Consumer + KafkaConsumer consumer = new KafkaConsumer<>(properties); + + // Subscribe Consumer to Our Topics + String topics = "test-topic"; + consumer.assign(Arrays.asList(new TopicPartition(topics, 1))); + + logger.info("Waiting for messages..."); + // Poll the data + while (true) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(1000)); + + for (ConsumerRecord record : records) { + logger.info("Value: " + record.value() + " -- Partition: " + record.partition()); + } + } + } + +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/assignsubscribe/SubscriberUsingSubscribe.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/assignsubscribe/SubscriberUsingSubscribe.java new file mode 100644 index 000000000000..2baff88394fe --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/assignsubscribe/SubscriberUsingSubscribe.java @@ -0,0 +1,47 @@ +package com.baeldung.kafka.assignsubscribe; + +import java.time.Duration; +import java.util.List; +import java.util.Properties; + +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class SubscriberUsingSubscribe { + + public static void main(String[] args) { + Logger logger = LoggerFactory.getLogger(SubscriberUsingSubscribe.class); + + // Create and Set Consumer Properties + Properties properties = new Properties(); + String bootstrapServer = "127.0.0.1:9092"; + String groupId = "baeldung-consumer-group"; + properties.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServer); + properties.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + properties.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + properties.setProperty(ConsumerConfig.GROUP_ID_CONFIG, groupId); + + // Create Kafka Consumer + KafkaConsumer consumer = new KafkaConsumer<>(properties); + + // Subscribe Consumer to Our Topics + String topics = "test-topic"; + consumer.subscribe(List.of(topics)); + + logger.info("Waiting for messages..."); + // Poll the data + while (true) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(1000)); + + for (ConsumerRecord record : records) { + logger.info("Value: " + record.value() + " -- Partition: " + record.partition()); + } + } + } + +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/fixed/async/KafkaConsumerService.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/fixed/async/KafkaConsumerService.java new file mode 100644 index 000000000000..f091cba4cffe --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/fixed/async/KafkaConsumerService.java @@ -0,0 +1,176 @@ +package com.baeldung.kafka.commitfailure.fixed.async; + +import java.time.Duration; +import java.util.Collection; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionException; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLong; +import java.util.stream.StreamSupport; + +import org.apache.kafka.clients.consumer.ConsumerRebalanceListener; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.clients.consumer.OffsetAndMetadata; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.errors.WakeupException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class KafkaConsumerService { + + private static final Logger log = LoggerFactory.getLogger(KafkaConsumerService.class); + private final KafkaConsumer consumer; + private final AtomicBoolean running = new AtomicBoolean(true); + private final ExecutorService workers; + private final Map committableOffsets = new ConcurrentHashMap<>(); + + public KafkaConsumerService(Properties consumerProps, String topic) { + this.consumer = new KafkaConsumer<>(consumerProps); + consumer.subscribe(List.of(topic), new ConsumerRebalanceListener() { + @Override + public void onPartitionsRevoked(Collection partitions) { + try { + commitOffsets(partitions); + } catch (Exception ex) { + log.error("Commit failed during rebalance", ex); + } finally { + partitions.forEach(committableOffsets::remove); + } + } + + @Override + public void onPartitionsAssigned(Collection partitions) { + partitions.forEach(committableOffsets::remove); + } + }); + workers = Executors.newVirtualThreadPerTaskExecutor(); + } + + public void consume() { + try { + while (running.get()) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); + if (records.isEmpty()) { + continue; + } + + List> futures = processAsync(records); + + try { + CompletableFuture.allOf(futures.toArray(CompletableFuture[]::new)) + .orTimeout(700L, TimeUnit.MILLISECONDS) + .join(); + } catch (CompletionException ex) { + log.error("Batch processing timed out or failed", ex); + } + + commitOffsets(); + } + } catch (WakeupException ex) { + if (running.get()) { + log.error("Error in the Kafka Consumer with exception", ex); + throw ex; + } + } finally { + commitOffsets(); + consumer.close(); + } + } + + public void shutdown() { + running.compareAndSet(true, false); + consumer.wakeup(); + try { + workers.shutdown(); + if (!workers.awaitTermination(60, TimeUnit.SECONDS)) { + workers.shutdownNow(); + } + } catch (InterruptedException ex) { + workers.shutdownNow(); + Thread.currentThread() + .interrupt(); + } + } + + private List> processAsync(ConsumerRecords records) { + return StreamSupport.stream(records.spliterator(), false) + .map(record -> CompletableFuture.runAsync(() -> simulateDBUpdate(record), workers) + .whenComplete((ignored, ex) -> { + if (ex == null) { + markComplete(record); + } else { + log.error("Failed offset and send to DLQ {} {} {}", record.offset(), record.key(), ex.getMessage()); + } + }) + .exceptionally(ex -> null)) + .toList(); + } + + private void simulateDBUpdate(ConsumerRecord record) { + try { + log.info("Simulating a db call - record key {} value {}", record.key(), record.value()); + Thread.sleep(300L); + } catch (InterruptedException ex) { + Thread.currentThread() + .interrupt(); + throw new RuntimeException(ex); + } + } + + private void markComplete(ConsumerRecord record) { + TopicPartition tp = new TopicPartition(record.topic(), record.partition()); + committableOffsets.computeIfAbsent(tp, k -> new AtomicLong(-1L)) + .accumulateAndGet(record.offset() + 1L, Math::max); + } + + private void commitOffsets() { + Map toCommit = new HashMap<>(); + + committableOffsets.forEach((tp, atomicOffset) -> { + long val = atomicOffset.get(); + if (val != -1L) { + toCommit.put(tp, new OffsetAndMetadata(val)); + } + }); + + if (toCommit.isEmpty()) { + return; + } + consumer.commitSync(toCommit); + toCommit.forEach((tp, meta) -> { + AtomicLong ref = committableOffsets.get(tp); + if (ref != null) { + ref.compareAndSet(meta.offset(), -1L); + } + }); + } + + private void commitOffsets(Collection partitions) { + Map toCommit = new HashMap<>(); + + partitions.forEach(tp -> { + AtomicLong ref = committableOffsets.get(tp); + if (ref != null) { + long val = ref.get(); + if (val != -1L) { + toCommit.put(tp, new OffsetAndMetadata(val)); + } + } + }); + + if (toCommit.isEmpty()) { + return; + } + consumer.commitSync(toCommit); + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/fixed/sequential/KafkaConsumerService.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/fixed/sequential/KafkaConsumerService.java new file mode 100644 index 000000000000..0d19d3817d47 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/fixed/sequential/KafkaConsumerService.java @@ -0,0 +1,61 @@ +package com.baeldung.kafka.commitfailure.fixed.sequential; + +import java.time.Duration; +import java.util.List; +import java.util.Properties; +import java.util.concurrent.atomic.AtomicBoolean; + +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.common.errors.WakeupException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class KafkaConsumerService { + + private static final Logger log = LoggerFactory.getLogger(KafkaConsumerService.class); + private final KafkaConsumer consumer; + private final AtomicBoolean running = new AtomicBoolean(true); + + public KafkaConsumerService(Properties consumerProps, String topic) { + this.consumer = new KafkaConsumer<>(consumerProps); + consumer.subscribe(List.of(topic)); + } + + public void consume() { + try { + while (running.get()) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); + if (records.isEmpty()) { + continue; + } + records.forEach(this::simulateDBUpdate); + consumer.commitSync(); + } + } catch (WakeupException ex) { + if (running.get()) { + log.error("Error in the Kafka Consumer with exception", ex); + throw ex; + } + } finally { + consumer.close(); + } + } + + public void shutdown() { + running.compareAndSet(true, false); + consumer.wakeup(); + } + + private void simulateDBUpdate(ConsumerRecord record) { + try { + log.info("Simulating a db call - record key {} value {}", record.key(), record.value()); + Thread.sleep(150L); + } catch (InterruptedException ex) { + Thread.currentThread() + .interrupt(); + throw new RuntimeException(ex); + } + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/sequential/KafkaConsumerService.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/sequential/KafkaConsumerService.java new file mode 100644 index 000000000000..63d08c339361 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/commitfailure/sequential/KafkaConsumerService.java @@ -0,0 +1,61 @@ +package com.baeldung.kafka.commitfailure.sequential; + +import java.time.Duration; +import java.util.List; +import java.util.Properties; +import java.util.concurrent.atomic.AtomicBoolean; + +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.common.errors.WakeupException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class KafkaConsumerService { + + private static final Logger log = LoggerFactory.getLogger(KafkaConsumerService.class); + private final KafkaConsumer consumer; + private final AtomicBoolean running = new AtomicBoolean(true); + + public KafkaConsumerService(Properties props, String topic) { + this.consumer = new KafkaConsumer<>(props); + consumer.subscribe(List.of(topic)); + } + + public void consume() { + try { + while (running.get()) { + ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); + if (records.isEmpty()) { + continue; + } + records.forEach(this::simulateDBCall); + consumer.commitSync(); + } + } catch (WakeupException ex) { + if (running.get()) { + log.error("Error in the Kafka Consumer with exception", ex); + throw ex; + } + } finally { + consumer.close(); + } + } + + public void shutdown() { + running.compareAndSet(true, false); + consumer.wakeup(); + } + + private void simulateDBCall(ConsumerRecord record) { + try { + log.info("Simulating a DB call - record key {} value {}", record.key(), record.value()); + Thread.sleep(150L); + } catch (InterruptedException ex) { + Thread.currentThread() + .interrupt(); + throw new RuntimeException(ex); + } + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/admin/ResetOffsetService.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/admin/ResetOffsetService.java new file mode 100644 index 000000000000..98bbfcac1cc4 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/admin/ResetOffsetService.java @@ -0,0 +1,83 @@ +package com.baeldung.kafka.resetoffset.admin; + +import org.apache.kafka.clients.admin.*; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.clients.consumer.OffsetAndMetadata; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.*; +import java.util.concurrent.ExecutionException; +import java.util.stream.Collectors; + +public class ResetOffsetService { + + private static final Logger log = LoggerFactory.getLogger(ResetOffsetService.class); + private final AdminClient adminClient; + + public ResetOffsetService(String bootstrapServers) { + this.adminClient = AdminClient.create(Map.of(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers)); + } + + public void reset(String topic, String consumerGroup) { + List partitions; + try { + partitions = fetchPartitions(topic); + } catch (ExecutionException | InterruptedException ex) { + log.error("Error in the fetching partitions with exception {}", ex.getMessage(), ex); + throw new RuntimeException(ex); + } + + Map earliestOffsets = fetchEarliestOffsets(partitions); + + try { + adminClient.alterConsumerGroupOffsets(consumerGroup, earliestOffsets) + .all() + .get(); + } catch (InterruptedException | ExecutionException ex) { + log.error("Error in the Kafka Consumer reset with exception {}", ex.getMessage(), ex); + throw new RuntimeException(ex); + } + } + + private List fetchPartitions(String topic) throws ExecutionException, InterruptedException { + return adminClient.describeTopics(List.of(topic)) + .values() + .get(topic) + .get() + .partitions() + .stream() + .map(p -> new TopicPartition(topic, p.partition())) + .toList(); + } + + private Map fetchEarliestOffsets(List partitions) { + Map offsetSpecs = partitions.stream() + .collect(Collectors.toMap(tp -> tp, tp -> OffsetSpec.earliest())); + + ListOffsetsResult offsetsResult = adminClient.listOffsets(offsetSpecs); + Map offsets = new HashMap<>(); + + partitions.forEach(tp -> { + long offset = Optional.ofNullable(offsetsResult.partitionResult(tp)) + .map(kafkaFuture -> { + try { + return kafkaFuture.get(); + } catch (InterruptedException | ExecutionException ex) { + log.error("Error in the Kafka Consumer reset with exception {}", ex.getMessage(), ex); + throw new RuntimeException(ex); + } + }) + .map(ListOffsetsResult.ListOffsetsResultInfo::offset) + .orElseThrow(() -> new RuntimeException("No offset result returned for partition " + tp)); + + offsets.put(tp, new OffsetAndMetadata(offset)); + }); + + return offsets; + } + + public void close() { + adminClient.close(); + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/consumer/KafkaConsumerService.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/consumer/KafkaConsumerService.java new file mode 100644 index 000000000000..a4e6afebc9dd --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/consumer/KafkaConsumerService.java @@ -0,0 +1,52 @@ +package com.baeldung.kafka.resetoffset.consumer; + +import org.apache.kafka.clients.consumer.*; +import org.apache.kafka.common.errors.WakeupException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.time.Duration; +import java.util.List; +import java.util.Properties; +import java.util.concurrent.atomic.AtomicBoolean; + +public class KafkaConsumerService { + + private static final Logger log = LoggerFactory.getLogger(KafkaConsumerService.class); + private final KafkaConsumer consumer; + private final AtomicBoolean running = new AtomicBoolean(true); + + public KafkaConsumerService(Properties consumerProps, String topic, long replayFromTimestampInEpoch) { + this.consumer = new KafkaConsumer<>(consumerProps); + + if (replayFromTimestampInEpoch > 0) { + consumer.subscribe(List.of(topic), new ReplayRebalanceListener(consumer, replayFromTimestampInEpoch)); + } else { + consumer.subscribe(List.of(topic)); + } + } + + public void start() { + try { + while (running.get()) { + ConsumerRecords records = consumer.poll(Duration.ofSeconds(1)); + records.forEach(record -> + log.info("topic={} partition={} offset={} key={} value={}", record.topic(), record.partition(), + record.offset(), record.key(), record.value())); + consumer.commitSync(); + } + } catch (WakeupException ex) { + if (running.get()) { + log.error("Error in the Kafka Consumer with exception {}", ex.getMessage(), ex); + throw ex; + } + } finally { + consumer.close(); + } + } + + public void shutdown() { + running.set(false); + consumer.wakeup(); + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/consumer/ReplayRebalanceListener.java b/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/consumer/ReplayRebalanceListener.java new file mode 100644 index 000000000000..abed553485ac --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafka/resetoffset/consumer/ReplayRebalanceListener.java @@ -0,0 +1,49 @@ +package com.baeldung.kafka.resetoffset.consumer; + +import org.apache.kafka.clients.consumer.ConsumerRebalanceListener; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.clients.consumer.OffsetAndTimestamp; +import org.apache.kafka.common.TopicPartition; + +import java.util.*; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.function.Function; +import java.util.stream.Collectors; + +public class ReplayRebalanceListener implements ConsumerRebalanceListener { + + private final KafkaConsumer consumer; + private final long replayFromTimeInEpoch; + private final AtomicBoolean seekDone = new AtomicBoolean(false); + + public ReplayRebalanceListener(KafkaConsumer consumer, long replayFromTimeInEpoch) { + this.consumer = consumer; + this.replayFromTimeInEpoch = replayFromTimeInEpoch; + } + + @Override + public void onPartitionsRevoked(Collection partitions) { + consumer.commitSync(); + } + + @Override + public void onPartitionsAssigned(Collection partitions) { + if (seekDone.get() || partitions.isEmpty()) { + return; + } + + Map partitionsTimestamp = partitions.stream() + .collect(Collectors.toMap(Function.identity(), tp -> replayFromTimeInEpoch)); + + Map offsets = consumer.offsetsForTimes(partitionsTimestamp); + + partitions.forEach(tp -> { + OffsetAndTimestamp offsetAndTimestamp = offsets.get(tp); + if (offsetAndTimestamp != null) { + consumer.seek(tp, offsetAndTimestamp.offset()); + } + }); + + seekDone.set(true); + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/CustomProcessingExceptionHandler.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/CustomProcessingExceptionHandler.java new file mode 100644 index 000000000000..c86896302120 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/CustomProcessingExceptionHandler.java @@ -0,0 +1,26 @@ +package com.baeldung.kafkastreams; + +import org.apache.kafka.streams.errors.ErrorHandlerContext; +import org.apache.kafka.streams.errors.ProcessingExceptionHandler; +import org.apache.kafka.streams.processor.api.Record; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Map; + +public class CustomProcessingExceptionHandler implements ProcessingExceptionHandler { + + private static final Logger log = LoggerFactory.getLogger(CustomProcessingExceptionHandler.class); + + @Override + public ProcessingHandlerResponse handle(ErrorHandlerContext errorHandlerContext, Record record, Exception ex) { + log.error("ProcessingExceptionHandler Error for record NodeId: {} | TaskId: {} | Key: {} | Value: {} | Exception: {}", + errorHandlerContext.processorNodeId(), errorHandlerContext.taskId(), record.key(), record.value(), ex.getMessage(), ex); + + return ProcessingHandlerResponse.CONTINUE; + } + + @Override + public void configure(Map configs) { + } +} \ No newline at end of file diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/CustomProductionExceptionHandler.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/CustomProductionExceptionHandler.java new file mode 100644 index 000000000000..2f48bf8e7a11 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/CustomProductionExceptionHandler.java @@ -0,0 +1,26 @@ +package com.baeldung.kafkastreams; + +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.streams.errors.ErrorHandlerContext; +import org.apache.kafka.streams.errors.ProductionExceptionHandler; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Map; + +public class CustomProductionExceptionHandler implements ProductionExceptionHandler { + + private static final Logger log = LoggerFactory.getLogger(UserSerializer.class); + + @Override + public ProductionExceptionHandlerResponse handle(ErrorHandlerContext context, ProducerRecord record, Exception exception) { + log.error("ProductionExceptionHandler Error producing record NodeId: {} | TaskId: {} | Topic: {} | Partition: {} | Exception: {}", + context.processorNodeId(), context.taskId(), record.topic(), record.partition(), exception.getMessage(), exception); + + return ProductionExceptionHandlerResponse.CONTINUE; + } + + @Override + public void configure(Map configs) { + } +} \ No newline at end of file diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/StreamExceptionHandler.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/StreamExceptionHandler.java new file mode 100644 index 000000000000..3be70538ebe7 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/StreamExceptionHandler.java @@ -0,0 +1,16 @@ +package com.baeldung.kafkastreams; + +import org.apache.kafka.streams.errors.StreamsUncaughtExceptionHandler; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class StreamExceptionHandler implements StreamsUncaughtExceptionHandler { + + private static final Logger log = LoggerFactory.getLogger(StreamExceptionHandler.class); + + @Override + public StreamThreadExceptionResponse handle(Throwable exception) { + log.error("Stream encountered fatal exception: {}", exception.getMessage(), exception); + return StreamThreadExceptionResponse.REPLACE_THREAD; + } +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/User.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/User.java new file mode 100644 index 000000000000..023634a12343 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/User.java @@ -0,0 +1,4 @@ +package com.baeldung.kafkastreams; + +public record User(String id, String name, String country) { +} diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserDeserializer.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserDeserializer.java new file mode 100644 index 000000000000..dd615305bc04 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserDeserializer.java @@ -0,0 +1,28 @@ +package com.baeldung.kafkastreams; + +import com.fasterxml.jackson.databind.ObjectMapper; + +import org.apache.kafka.common.serialization.Deserializer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; + +public class UserDeserializer implements Deserializer { + private static final Logger log = LoggerFactory.getLogger(UserDeserializer.class); + private final ObjectMapper mapper = new ObjectMapper(); + + @Override + public User deserialize(String topic, byte[] bytes) { + if (bytes == null || bytes.length == 0) { + return null; + } + try { + return mapper.readValue(bytes, User.class); + } catch (IOException ex) { + log.error("Error deserializing the message {} for topic {} error message {}", bytes, topic, ex.getMessage(), ex); + throw new RuntimeException(ex); + } + } +} + diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserSerde.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserSerde.java new file mode 100644 index 000000000000..e4a57e7d781f --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserSerde.java @@ -0,0 +1,10 @@ +package com.baeldung.kafkastreams; + +import org.apache.kafka.common.serialization.Serdes; + +public class UserSerde extends Serdes.WrapperSerde { + public UserSerde() { + super(new UserSerializer(), new UserDeserializer()); + } +} + diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserSerializer.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserSerializer.java new file mode 100644 index 000000000000..525d7e529a81 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserSerializer.java @@ -0,0 +1,27 @@ +package com.baeldung.kafkastreams; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.kafka.common.serialization.Serializer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class UserSerializer implements Serializer { + private static final Logger log = LoggerFactory.getLogger(UserSerializer.class); + private final ObjectMapper mapper = new ObjectMapper(); + + @Override + public byte[] serialize(String topic, User user) { + if (user == null) { + return null; + } + + try { + return mapper.writeValueAsBytes(user); + } catch (JsonProcessingException ex) { + log.error("Error deserializing the user {} with exception {}", user, ex.getMessage(), ex); + throw new RuntimeException(ex); + } + } +} + diff --git a/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserStreamService.java b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserStreamService.java new file mode 100644 index 000000000000..7a5c8cd750c1 --- /dev/null +++ b/apache-kafka-4/src/main/java/com/baeldung/kafkastreams/UserStreamService.java @@ -0,0 +1,73 @@ +package com.baeldung.kafkastreams; + +import org.apache.kafka.common.serialization.Serdes; +import org.apache.kafka.streams.*; +import org.apache.kafka.streams.errors.LogAndContinueExceptionHandler; +import org.apache.kafka.streams.kstream.*; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Objects; +import java.util.Properties; +import java.util.UUID; + +public class UserStreamService { + private static final Logger log = LoggerFactory.getLogger(UserStreamService.class); + private KafkaStreams kafkaStreams; + + public void start(String bootstrapServer) { + StreamsBuilder builder = new StreamsBuilder(); + KStream userStream = builder.stream( + "user-topic", + Consumed.with(Serdes.String(), new UserSerde()) + ); + + KTable usersPerCountry = userStream + .filter((key, user) -> + Objects.nonNull(user) && user.country() != null && !user.country().isEmpty()) + .groupBy((key, user) -> user.country(), Grouped.with(Serdes.String(), + new UserSerde())) + .count(Materialized.as("users_per_country_store")); + + usersPerCountry.toStream() + .peek((country, count) -> log.info("Aggregated for country {} with count {}",country, count)) + .to("users_per_country", Produced.with(Serdes.String(), Serdes.Long())); + + Properties props = getStreamProperties(bootstrapServer); + kafkaStreams = new KafkaStreams(builder.build(), props); + kafkaStreams.setUncaughtExceptionHandler(new StreamExceptionHandler()); + + Runtime.getRuntime().addShutdownHook(new Thread(kafkaStreams::close)); + + kafkaStreams.start(); + } + + public void stop() { + if (kafkaStreams != null) { + kafkaStreams.close(); + } + } + + public void cleanUp() { + if (kafkaStreams != null) { + kafkaStreams.cleanUp(); + } + } + + private static Properties getStreamProperties(String bootstrapServer) { + Properties props = new Properties(); + props.put(StreamsConfig.APPLICATION_ID_CONFIG, "user-country-aggregator" + UUID.randomUUID()); + props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServer); + props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass()); + props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass()); + props.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG, + LogAndContinueExceptionHandler.class); + props.put(StreamsConfig.PROCESSING_EXCEPTION_HANDLER_CLASS_CONFIG, + CustomProcessingExceptionHandler.class); + props.put(StreamsConfig.DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG, + CustomProductionExceptionHandler.class); + + return props; + } +} + diff --git a/apache-kafka-4/src/main/resources/docker-compose-idempotence-benchmark.yml b/apache-kafka-4/src/main/resources/docker-compose-idempotence-benchmark.yml new file mode 100644 index 000000000000..7fd15ce9c612 --- /dev/null +++ b/apache-kafka-4/src/main/resources/docker-compose-idempotence-benchmark.yml @@ -0,0 +1,51 @@ +services: + kafka-1: + image: apache/kafka:3.9.0 + container_name: kafka-1 + ports: + - "29092:9092" + environment: + KAFKA_NODE_ID: 1 + KAFKA_PROCESS_ROLES: "broker,controller" + KAFKA_LISTENERS: "PLAINTEXT://:19092,CONTROLLER://:19093,EXTERNAL://:9092" + KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka-1:19092,EXTERNAL://localhost:29092" + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT" + KAFKA_INTER_BROKER_LISTENER_NAME: "PLAINTEXT" + KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER" + KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka-1:19093,2@kafka-2:19093,3@kafka-3:19093" + KAFKA_CLUSTER_ID: "71bb1db7-7a3d-41c6-8453-02c67e6bd2d0" + KAFKA_MIN_INSYNC_REPLICAS: 2 + + kafka-2: + image: apache/kafka:3.9.0 + container_name: kafka-2 + ports: + - "39092:9092" + environment: + KAFKA_NODE_ID: 2 + KAFKA_PROCESS_ROLES: "broker,controller" + KAFKA_LISTENERS: "PLAINTEXT://:19092,CONTROLLER://:19093,EXTERNAL://:9092" + KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka-2:19092,EXTERNAL://localhost:39092" + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT" + KAFKA_INTER_BROKER_LISTENER_NAME: "PLAINTEXT" + KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER" + KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka-1:19093,2@kafka-2:19093,3@kafka-3:19093" + KAFKA_CLUSTER_ID: "71bb1db7-7a3d-41c6-8453-02c67e6bd2d0" + KAFKA_MIN_INSYNC_REPLICAS: 2 + + kafka-3: + image: apache/kafka:3.9.0 + container_name: kafka-3 + ports: + - "49092:9092" + environment: + KAFKA_NODE_ID: 3 + KAFKA_PROCESS_ROLES: "broker,controller" + KAFKA_LISTENERS: "PLAINTEXT://:19092,CONTROLLER://:19093,EXTERNAL://:9092" + KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka-3:19092,EXTERNAL://localhost:49092" + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT" + KAFKA_INTER_BROKER_LISTENER_NAME: "PLAINTEXT" + KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER" + KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka-1:19093,2@kafka-2:19093,3@kafka-3:19093" + KAFKA_CLUSTER_ID: "71bb1db7-7a3d-41c6-8453-02c67e6bd2d0" + KAFKA_MIN_INSYNC_REPLICAS: 2 \ No newline at end of file diff --git a/apache-kafka-4/src/main/resources/logback.xml b/apache-kafka-4/src/main/resources/logback.xml new file mode 100644 index 000000000000..eacc629455a3 --- /dev/null +++ b/apache-kafka-4/src/main/resources/logback.xml @@ -0,0 +1,14 @@ + + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n + + + + + + + + + + \ No newline at end of file diff --git a/apache-kafka-4/src/test/java/com/baeldung/kafka/admin/ResetOffsetServiceLiveTest.java b/apache-kafka-4/src/test/java/com/baeldung/kafka/admin/ResetOffsetServiceLiveTest.java new file mode 100644 index 000000000000..6bdc9c4875dd --- /dev/null +++ b/apache-kafka-4/src/test/java/com/baeldung/kafka/admin/ResetOffsetServiceLiveTest.java @@ -0,0 +1,137 @@ +package com.baeldung.kafka.admin; + +import org.apache.kafka.clients.admin.*; +import org.apache.kafka.clients.consumer.*; +import org.apache.kafka.clients.producer.*; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; +import org.junit.jupiter.api.*; + +import org.testcontainers.containers.KafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; + +import java.time.Duration; +import java.util.*; +import java.util.concurrent.ExecutionException; + +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.testcontainers.shaded.org.awaitility.Awaitility.await; + +import com.baeldung.kafka.resetoffset.admin.ResetOffsetService; + +@Testcontainers +class ResetOffsetServiceLiveTest { + + @Container + private static final KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.9.0")); + private static ResetOffsetService resetService; + private static AdminClient testAdminClient; + + @BeforeAll + static void startKafka() { + KAFKA_CONTAINER.start(); + resetService = new ResetOffsetService(KAFKA_CONTAINER.getBootstrapServers()); + + Properties props = new Properties(); + props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + testAdminClient = AdminClient.create(props); + } + + @AfterAll + static void stopKafka() { + testAdminClient.close(); + resetService.close(); + KAFKA_CONTAINER.stop(); + } + + @Test + void givenMessagesAreConsumed_whenOffsetIsReset_thenOffsetIsSetToEarliest() { + KafkaProducer producer = new KafkaProducer<>(getProducerConfig()); + producer.send(new ProducerRecord<>("test-topic-1", "msg-1")); + producer.send(new ProducerRecord<>("test-topic-1", "msg-2")); + producer.flush(); + + KafkaConsumer consumer = new KafkaConsumer<>(getConsumerConfig("test-group-1")); + consumer.subscribe(List.of("test-topic-1")); + + int consumed = 0; + while (consumed < 2) { + ConsumerRecords records = consumer.poll(Duration.ofSeconds(1)); + consumed += records.count(); + } + + consumer.commitSync(); + consumer.close(); + + await().atMost(5, SECONDS) + .pollInterval(Duration.ofMillis(300)) + .untilAsserted(() -> assertEquals(2L, fetchCommittedOffset("test-group-1"))); + + resetService.reset("test-topic-1", "test-group-1"); + + await().atMost(5, SECONDS) + .pollInterval(Duration.ofMillis(300)) + .untilAsserted(() -> assertEquals(0L, fetchCommittedOffset("test-group-1"))); + } + + @Test + void givenConsumerIsStillActive_whenOffsetResetIsCalled_thenThrowRuntimeException_NoOffsetReset() { + KafkaProducer producer = new KafkaProducer<>(getProducerConfig()); + producer.send(new ProducerRecord<>("test-topic-2", "msg-1")); + producer.send(new ProducerRecord<>("test-topic-2", "msg-2")); + producer.flush(); + + KafkaConsumer consumer = new KafkaConsumer<>(getConsumerConfig("test-group-2")); + consumer.subscribe(List.of("test-topic-2")); + + int consumed = 0; + while (consumed < 2) { + ConsumerRecords records = consumer.poll(Duration.ofSeconds(1)); + consumed += records.count(); + } + consumer.commitSync(); + + assertThrows(RuntimeException.class, () -> resetService.reset("test-topic-2", "test-group-2")); + + await().atMost(5, SECONDS) + .pollInterval(Duration.ofMillis(300)) + .untilAsserted(() -> assertEquals(2L, fetchCommittedOffset("test-group-2"))); + } + + private static Properties getProducerConfig() { + Properties producerProperties = new Properties(); + producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + + return producerProperties; + } + + private static Properties getConsumerConfig(String groupId) { + Properties consumerProperties = new Properties(); + consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, groupId); + consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + consumerProperties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false"); + + return consumerProperties; + } + + private long fetchCommittedOffset(String groupId) throws ExecutionException, InterruptedException { + Map offsets = testAdminClient.listConsumerGroupOffsets(groupId) + .partitionsToOffsetAndMetadata() + .get(); + + return offsets.values() + .iterator() + .next() + .offset(); + } +} diff --git a/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/fixed/async/KafkaConsumerServiceLiveTest.java b/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/fixed/async/KafkaConsumerServiceLiveTest.java new file mode 100644 index 000000000000..a73fc99474c9 --- /dev/null +++ b/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/fixed/async/KafkaConsumerServiceLiveTest.java @@ -0,0 +1,97 @@ +package com.baeldung.kafka.commitfailure.fixed.async; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.TimeUnit; + +import org.apache.kafka.clients.admin.AdminClient; +import org.apache.kafka.clients.admin.AdminClientConfig; +import org.apache.kafka.clients.admin.ListConsumerGroupOffsetsResult; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.OffsetAndMetadata; +import org.apache.kafka.clients.producer.KafkaProducer; +import org.apache.kafka.clients.producer.ProducerConfig; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; +import org.awaitility.Awaitility; +import org.junit.jupiter.api.Test; +import org.testcontainers.containers.KafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; + +@Testcontainers +public class KafkaConsumerServiceLiveTest { + + @Container + private static final KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.9.0")); + + @Test + void givenProducerMessagesAreSent_whenConsumerIsRunning_thenConsumerOffsetIsCommitted() { + KafkaConsumerService kafkaConsumerService = new KafkaConsumerService(getConsumerConfig(), "test-topic"); + Thread th = new Thread(kafkaConsumerService::consume); + th.start(); + + try (KafkaProducer producer = new KafkaProducer<>(getProducerConfig())) { + for (int num = 0; num < 100; num++) { + producer.send(new ProducerRecord<>("test-topic", "x1" + num, "test" + num)); + } + producer.flush(); + } + + Awaitility.await() + .atMost(30, TimeUnit.SECONDS) + .pollInterval(2, TimeUnit.SECONDS) + .untilAsserted(() -> { + TopicPartition topicPartition = new TopicPartition("test-topic", 0); + Map committedOffsets; + + try (AdminClient adminClient = AdminClient.create(getAdminProps())) { + ListConsumerGroupOffsetsResult result = adminClient.listConsumerGroupOffsets("consumer-app"); + committedOffsets = result.partitionsToOffsetAndMetadata() + .get(); + } + + assertNotNull(committedOffsets); + assertNotNull(committedOffsets.get(topicPartition)); + assertEquals(100L, committedOffsets.get(topicPartition) + .offset()); + }); + + kafkaConsumerService.shutdown(); + } + + private static Properties getProducerConfig() { + Properties producerProperties = new Properties(); + producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + + return producerProperties; + } + + private static Properties getConsumerConfig() { + Properties consumerProperties = new Properties(); + consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, "consumer-app"); + consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + consumerProperties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false"); + consumerProperties.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, 30); + consumerProperties.put(ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG, 1000); + + return consumerProperties; + } + + private static Properties getAdminProps() { + Properties props = new Properties(); + props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + return props; + } +} diff --git a/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/fixed/sequential/KafkaConsumerServiceLiveTest.java b/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/fixed/sequential/KafkaConsumerServiceLiveTest.java new file mode 100644 index 000000000000..d5e9c4e45e7d --- /dev/null +++ b/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/fixed/sequential/KafkaConsumerServiceLiveTest.java @@ -0,0 +1,95 @@ +package com.baeldung.kafka.commitfailure.fixed.sequential; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.TimeUnit; + +import org.apache.kafka.clients.admin.AdminClient; +import org.apache.kafka.clients.admin.AdminClientConfig; +import org.apache.kafka.clients.admin.ListConsumerGroupOffsetsResult; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.OffsetAndMetadata; +import org.apache.kafka.clients.producer.KafkaProducer; +import org.apache.kafka.clients.producer.ProducerConfig; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; +import org.awaitility.Awaitility; +import org.junit.jupiter.api.Test; +import org.testcontainers.containers.KafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; + +@Testcontainers +public class KafkaConsumerServiceLiveTest { + + @Container + private static final KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.9.0")); + + @Test + void givenProducerMessageIsSent_whenConsumerIsRunning_thenConsumerOffsetIsCommitted() { + KafkaConsumerService kafkaConsumerService = new KafkaConsumerService(getConsumerConfig(), "test-topic"); + Thread th = new Thread(kafkaConsumerService::consume); + th.start(); + + try (KafkaProducer producer = new KafkaProducer<>(getProducerConfig())) { + producer.send(new ProducerRecord<>("test-topic", "x1", "test")); + producer.flush(); + } + + Awaitility.await() + .atMost(30, TimeUnit.SECONDS) + .pollInterval(2, TimeUnit.SECONDS) + .untilAsserted(() -> { + TopicPartition topicPartition = new TopicPartition("test-topic", 0); + Map committedOffsets; + + try (AdminClient adminClient = AdminClient.create(getAdminProps())) { + ListConsumerGroupOffsetsResult result = adminClient.listConsumerGroupOffsets("consumer-app"); + committedOffsets = result.partitionsToOffsetAndMetadata() + .get(); + } + + assertNotNull(committedOffsets); + assertNotNull(committedOffsets.get(topicPartition)); + assertEquals(1L, committedOffsets.get(topicPartition) + .offset()); + }); + + kafkaConsumerService.shutdown(); + } + + private static Properties getProducerConfig() { + Properties producerProperties = new Properties(); + producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + + return producerProperties; + } + + private static Properties getConsumerConfig() { + Properties consumerProperties = new Properties(); + consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, "consumer-app"); + consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + consumerProperties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false"); + consumerProperties.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, 1); + consumerProperties.put(ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG, 300); + + return consumerProperties; + } + + private static Properties getAdminProps() { + Properties props = new Properties(); + props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + return props; + } +} diff --git a/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/sequential/KafkaConsumerServiceLiveTest.java b/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/sequential/KafkaConsumerServiceLiveTest.java new file mode 100644 index 000000000000..4f42f9ab9759 --- /dev/null +++ b/apache-kafka-4/src/test/java/com/baeldung/kafka/commitfailure/sequential/KafkaConsumerServiceLiveTest.java @@ -0,0 +1,78 @@ +package com.baeldung.kafka.commitfailure.sequential; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.util.Properties; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicReference; + +import org.apache.kafka.clients.consumer.CommitFailedException; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.producer.KafkaProducer; +import org.apache.kafka.clients.producer.ProducerConfig; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; +import org.junit.jupiter.api.Test; +import org.testcontainers.containers.KafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; + +@Testcontainers +public class KafkaConsumerServiceLiveTest { + + @Container + private static final KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.9.0")); + + @Test + void givenProducerMessageIsSent_whenConsumerIsRunning_thenConsumerThrowsCommitFailedException() throws InterruptedException { + CountDownLatch countDownLatch = new CountDownLatch(1); + AtomicReference uncaughtException = new AtomicReference<>(); + + KafkaConsumerService kafkaConsumerService = new KafkaConsumerService(getConsumerConfig(), "test-topic"); + Thread th = new Thread(kafkaConsumerService::consume); + th.setUncaughtExceptionHandler((thread, ex) -> { + uncaughtException.set(ex); + countDownLatch.countDown(); + }); + + th.start(); + + try (KafkaProducer producer = new KafkaProducer<>(getProducerConfig())) { + producer.send(new ProducerRecord<>("test-topic", "x1", "test1")); + producer.flush(); + } + + assertTrue(countDownLatch.await(30, TimeUnit.SECONDS)); + assertThat(uncaughtException.get()).isNotNull() + .isInstanceOf(CommitFailedException.class); + + kafkaConsumerService.shutdown(); + } + + private static Properties getProducerConfig() { + Properties producerProperties = new Properties(); + producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + + return producerProperties; + } + + private static Properties getConsumerConfig() { + Properties consumerProperties = new Properties(); + consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, "consumer-app"); + consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + consumerProperties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false"); + consumerProperties.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, 1); + consumerProperties.put(ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG, 50); + + return consumerProperties; + } +} diff --git a/apache-kafka-4/src/test/java/com/baeldung/kafka/consumer/KafkaConsumerServiceLiveTest.java b/apache-kafka-4/src/test/java/com/baeldung/kafka/consumer/KafkaConsumerServiceLiveTest.java new file mode 100644 index 000000000000..234e18c82f66 --- /dev/null +++ b/apache-kafka-4/src/test/java/com/baeldung/kafka/consumer/KafkaConsumerServiceLiveTest.java @@ -0,0 +1,159 @@ +package com.baeldung.kafka.consumer; + +import org.apache.kafka.clients.consumer.*; +import org.apache.kafka.clients.producer.*; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; +import org.awaitility.Awaitility; +import org.junit.jupiter.api.*; +import org.testcontainers.containers.KafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; + +import java.time.Duration; +import java.util.*; +import java.util.concurrent.TimeUnit; + +import static org.junit.jupiter.api.Assertions.*; + +import com.baeldung.kafka.resetoffset.consumer.KafkaConsumerService; + +@Testcontainers +public class KafkaConsumerServiceLiveTest { + @Container + private static final KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.9.0")); + + @BeforeAll + static void setup() { + KAFKA_CONTAINER.start(); + } + + @AfterAll + static void cleanup() { + KAFKA_CONTAINER.stop(); + } + + @Test + void givenConsumerReplayIsEnabled_whenReplayTimestampIsProvided_thenConsumesFromTimestamp() { + KafkaProducer producer = new KafkaProducer<>(getProducerConfig()); + long firstMsgTs = System.currentTimeMillis(); + producer.send(new ProducerRecord<>("test-topic-1", 0, firstMsgTs, "x1", "test1")); + producer.flush(); + + long baseTs = System.currentTimeMillis(); + + long secondMsgTs = baseTs + 1L; + producer.send(new ProducerRecord<>("test-topic-1", 0, secondMsgTs, "x2", "test2")); + producer.flush(); + + KafkaConsumerService kafkaConsumerService = new KafkaConsumerService(getConsumerConfig("test-group-1"), "test-topic-1", baseTs); + new Thread(kafkaConsumerService::start).start(); + + Awaitility.await() + .atMost(45, TimeUnit.SECONDS) + .pollInterval(1, TimeUnit.SECONDS) + .untilAsserted(() -> { + List consumed = consumeFromCommittedOffset("test-topic-1", "test-group-1"); + assertEquals(0, consumed.size()); + assertFalse(consumed.contains("test1")); + assertFalse(consumed.contains("test2")); + }); + + kafkaConsumerService.shutdown(); + } + + @Test + void givenProducerMessagesSent_WhenConsumerIsRunningWithReplayDisabled_ThenConsumesLatestOffset() { + KafkaProducer producer = new KafkaProducer<>(getProducerConfig()); + producer.send(new ProducerRecord<>("test-topic-2", "x3", "test3")); + producer.send(new ProducerRecord<>("test-topic-2", "x4", "test4")); + producer.flush(); + + KafkaConsumerService service = new KafkaConsumerService(getConsumerConfig("test-group-2"), + "test-topic-2", 0L); + new Thread(service::start).start(); + + Awaitility.await() + .atMost(45, TimeUnit.SECONDS) + .pollInterval(1, TimeUnit.SECONDS) + .untilAsserted(() -> { + List consumed = consumeFromCommittedOffset("test-topic-2", "test-group-2"); + assertEquals(0, consumed.size()); + assertFalse(consumed.contains("test3")); + assertFalse(consumed.contains("test4")); + }); + + service.shutdown(); + } + + @Test + void givenConsumerWithReplayedDisabledRuns_whenReplayIsEnabled_WhenTimestampProvided_ThenConsumesFromTimestamp() throws InterruptedException { + KafkaProducer producer = new KafkaProducer<>(getProducerConfig()); + producer.send(new ProducerRecord<>("test-topic-3", "x5", "test5")); + producer.flush(); + + KafkaConsumerService service1 = new KafkaConsumerService(getConsumerConfig("test-group-3"), + "test-topic-3", 0L); + new Thread(service1::start).start(); + Thread.sleep(5000); + service1.shutdown(); + + producer.send(new ProducerRecord<>("test-topic-3", "x6", "test6")); + producer.flush(); + + KafkaConsumerService service2 = new KafkaConsumerService(getConsumerConfig("test-group-3"), + "test-topic-3", 0L); + new Thread(service2::start).start(); + + Awaitility.await() + .atMost(45, TimeUnit.SECONDS) + .pollInterval(1, TimeUnit.SECONDS) + .untilAsserted(() -> { + List consumed = consumeFromCommittedOffset("test-topic-3", "test-group-3"); + assertEquals(0, consumed.size()); + assertFalse(consumed.contains("test5")); + assertFalse(consumed.contains("test6")); + assertFalse(consumed.contains("test6")); + }); + + service2.shutdown(); + } + + private List consumeFromCommittedOffset(String topic, String groupId) { + List values = new ArrayList<>(); + + try (KafkaConsumer consumer = new KafkaConsumer<>(getConsumerConfig(groupId))) { + consumer.subscribe(Collections.singleton(topic)); + + ConsumerRecords records = consumer.poll(Duration.ofSeconds(2)); + for (ConsumerRecord r : records) { + values.add(r.value()); + } + } + + return values; + } + + private static Properties getProducerConfig() { + Properties producerProperties = new Properties(); + producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + + return producerProperties; + } + + private static Properties getConsumerConfig(String groupId) { + Properties consumerProperties = new Properties(); + consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, groupId); + consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest"); + consumerProperties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false"); + + + return consumerProperties; + } +} diff --git a/apache-kafka-4/src/test/java/com/baeldung/kafkastreams/UserStreamLiveTest.java b/apache-kafka-4/src/test/java/com/baeldung/kafkastreams/UserStreamLiveTest.java new file mode 100644 index 000000000000..e2b324bebe17 --- /dev/null +++ b/apache-kafka-4/src/test/java/com/baeldung/kafkastreams/UserStreamLiveTest.java @@ -0,0 +1,176 @@ +package com.baeldung.kafkastreams; + +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.clients.producer.KafkaProducer; +import org.apache.kafka.clients.producer.ProducerConfig; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.serialization.*; +import org.junit.jupiter.api.*; +import org.testcontainers.containers.KafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.shaded.org.awaitility.Awaitility; +import org.testcontainers.utility.DockerImageName; + +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.List; +import java.util.Map; +import java.util.Properties; +import java.util.UUID; +import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; +import java.util.stream.StreamSupport; + +import static org.junit.jupiter.api.Assertions.*; + +// This live test requires a Docker Daemon running so that a kafka container can be created + +@Testcontainers +class UserStreamLiveTest { + + @Container + private static final KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.9.0")); + private static KafkaProducer producer; + private static KafkaConsumer consumer; + + private static UserStreamService streamService; + + @BeforeAll + static void setup() { + KAFKA_CONTAINER.start(); + streamService = new UserStreamService(); + producer = new KafkaProducer<>(getProducerConfig()); + consumer = new KafkaConsumer<>(getConsumerConfig()); + new Thread(() -> streamService.start(KAFKA_CONTAINER.getBootstrapServers())).start(); + } + + @AfterAll + static void destroy() { + producer.flush(); + producer.close(); + consumer.close(); + streamService.stop(); + streamService.cleanUp(); + KAFKA_CONTAINER.stop(); + } + + @Test + void givenValidUserIsSent_whenStreamServiceStarts_thenAggregatedCountIsSent() { + producer.send(new ProducerRecord<>("user-topic", "x1", new User("1", "user1", "US"))); + producer.send(new ProducerRecord<>("user-topic", "x2", new User("2", "user2", "DE"))); + consumer.subscribe(List.of("users_per_country")); + + Awaitility.await() + .atMost(45, TimeUnit.SECONDS) + .pollInterval(Duration.ofSeconds(1)) + .untilAsserted(() -> { + ConsumerRecords records = consumer.poll(Duration.ofMillis(500)); + Map counts = StreamSupport.stream(records.spliterator(), false) + .collect(Collectors.toMap(ConsumerRecord::key, ConsumerRecord::value)); + + assertTrue(counts.containsKey("US")); + assertTrue(counts.containsKey("DE")); + assertEquals(1L, counts.get("US")); + assertEquals(1L, counts.get("DE")); + }); + } + + @Test + void givenValidAndNullUserIsSent_whenStreamServiceIsRunning_thenAggregatedCount() { + producer.send(new ProducerRecord<>("user-topic", "x3", new User("3", "user3", "IE"))); + producer.send(new ProducerRecord<>("user-topic", "x4", null)); + + consumer.subscribe(List.of("users_per_country")); + + Awaitility.await() + .atMost(30, TimeUnit.SECONDS) + .pollInterval(Duration.ofSeconds(1)) + .untilAsserted(() -> { + ConsumerRecords records = consumer.poll(Duration.ofMillis(500)); + Map counts = StreamSupport.stream(records.spliterator(), false) + .collect(Collectors.toMap(ConsumerRecord::key, ConsumerRecord::value)); + + assertTrue(counts.containsKey("IE")); + assertEquals(1L, counts.get("IE")); + }); + } + + @Test + void givenInvalidUserIsSent_whenStreamServiceIsRunning_thenAggregatedCountIsEmpty() { + Properties props = new Properties(); + props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class); + KafkaProducer producer = new KafkaProducer<>(props); + + byte[] invalidJson = "{ invalid json".getBytes(StandardCharsets.UTF_8); + producer.send(new ProducerRecord<>("user-topic", "x5", invalidJson)); + + consumer.subscribe(List.of("users_per_country")); + Awaitility.await() + .atMost(30, TimeUnit.SECONDS) + .pollInterval(Duration.ofSeconds(1)) + .untilAsserted(() -> { + ConsumerRecords records = consumer.poll(Duration.ofMillis(500)); + assertTrue(records.isEmpty()); + }); + } + + @Test + void givenEmptyUserJsonIsSent_whenStreamServiceIsRunning_thenAggregatedCountIsEmpty() { + Properties props = new Properties(); + props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class); + KafkaProducer producer = new KafkaProducer<>(props); + byte[] emptyJson = "".getBytes(StandardCharsets.UTF_8); + producer.send(new ProducerRecord<>("user-topic", "x6", emptyJson)); + + consumer.subscribe(List.of("users_per_country")); + Awaitility.await() + .atMost(30, TimeUnit.SECONDS) + .pollInterval(Duration.ofSeconds(1)) + .untilAsserted(() -> { + ConsumerRecords records = consumer.poll(Duration.ofMillis(500)); + assertTrue(records.isEmpty()); + }); + } + + @Test + void givenInvalidCountryUserIsSent_whenStreamServiceIsRunning_thenNoAggregatedCount() { + producer.send(new ProducerRecord<>("user-topic", "x7", new User("7", "user7", null))); + consumer.subscribe(List.of("users_per_country")); + + Awaitility.await() + .atMost(30, TimeUnit.SECONDS) + .pollInterval(Duration.ofSeconds(1)) + .untilAsserted(() -> { + ConsumerRecords records = consumer.poll(Duration.ofMillis(500)); + assertTrue(records.isEmpty()); + }); + } + + private static Properties getProducerConfig() { + Properties producerProperties = new Properties(); + producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, UserSerializer.class); + + return producerProperties; + } + + private static Properties getConsumerConfig() { + Properties props = new Properties(); + props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); + props.put(ConsumerConfig.GROUP_ID_CONFIG, "test-group-" + UUID.randomUUID()); + props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); + props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, LongDeserializer.class.getName()); + props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + + return props; + } +} diff --git a/apache-kafka-4/src/test/resources/docker/docker-compose.yml b/apache-kafka-4/src/test/resources/docker/docker-compose.yml new file mode 100644 index 000000000000..665e6c3119f2 --- /dev/null +++ b/apache-kafka-4/src/test/resources/docker/docker-compose.yml @@ -0,0 +1,23 @@ +version: "3.8" + +services: + kafka: + image: confluentinc/cp-kafka:7.9.0 + hostname: kafka + container_name: kafka + ports: + - "9092:9092" + - "9101:9101" + expose: + - '29092' + environment: + KAFKA_NODE_ID: 1 + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT' + KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092' + KAFKA_LISTENERS: 'PLAINTEXT://kafka:29092,CONTROLLER://kafka:29093,PLAINTEXT_HOST://0.0.0.0:9092' + KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 + KAFKA_PROCESS_ROLES: 'broker,controller' + KAFKA_CONTROLLER_QUORUM_VOTERS: '1@kafka:29093' + KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT' + KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER' + CLUSTER_ID: 'MkU3OEVBNTcwNTJENDM2Qk' diff --git a/apache-kafka-4/src/test/resources/logback.xml b/apache-kafka-4/src/test/resources/logback.xml new file mode 100644 index 000000000000..7175fc84fd5c --- /dev/null +++ b/apache-kafka-4/src/test/resources/logback.xml @@ -0,0 +1,12 @@ + + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n + + + + + + + + \ No newline at end of file diff --git a/apache-kafka/README.md b/apache-kafka/README.md index b02fd672e94e..f43d51c20cd7 100644 --- a/apache-kafka/README.md +++ b/apache-kafka/README.md @@ -2,17 +2,5 @@ This module contains articles about Apache Kafka. -### Relevant articles -- [Kafka Streams vs. Kafka Consumer](https://www.baeldung.com/java-kafka-streams-vs-kafka-consumer) -- [Introduction to Kafka Connectors](https://www.baeldung.com/kafka-connectors-guide) -- [Exactly Once Processing in Kafka with Java](https://www.baeldung.com/kafka-exactly-once) -- [Custom Serializers in Apache Kafka](https://www.baeldung.com/kafka-custom-serializer) -- [Read Data From the Beginning Using Kafka Consumer API](https://www.baeldung.com/java-kafka-consumer-api-read) -- [Add Custom Headers to a Kafka Message](https://www.baeldung.com/java-kafka-custom-headers) -- [bootstrap-server in Kafka Configuration](https://www.baeldung.com/java-kafka-bootstrap-server) -- [Ensuring Message Ordering in Kafka: Strategies and Configurations](https://www.baeldung.com/kafka-message-ordering) -- [Is a Key Required as Part of Sending Messages to Kafka?](https://www.baeldung.com/java-kafka-message-key) -- [Commit Offsets in Kafka](https://www.baeldung.com/kafka-commit-offsets) - ##### Building the project You can build the project from the command line using: *mvn clean install*, or in an IDE. diff --git a/apache-kafka/pom.xml b/apache-kafka/pom.xml index 8c7d82e962b2..1463fc76b34d 100644 --- a/apache-kafka/pom.xml +++ b/apache-kafka/pom.xml @@ -1,7 +1,7 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 apache-kafka apache-kafka @@ -101,7 +101,6 @@ ${lombok.version} provided - diff --git a/apache-libraries-2/README.md b/apache-libraries-2/README.md deleted file mode 100644 index 036fa5f772e6..000000000000 --- a/apache-libraries-2/README.md +++ /dev/null @@ -1,13 +0,0 @@ -## Relevant Articles -- [Add Camel Route at Runtime in Java](https://www.baeldung.com/java-camel-dynamic-route) -- [Logging in Apache Camel](https://www.baeldung.com/java-apache-camel-logging) -- [How to Handle Default Values in Avro](https://www.baeldung.com/java-avro-default-values) -- [How to Send a Post Request in Camel](https://www.baeldung.com/java-apache-camel-send-post-request) -- [Introduction to Apache Beam](https://www.baeldung.com/apache-beam) -- [Introduction to Apache Pulsar](https://www.baeldung.com/apache-pulsar) -- [Introduction to Apache Curator](https://www.baeldung.com/apache-curator) -- [Intro to Apache BVal](https://www.baeldung.com/apache-bval) -- [Building a Microservice with Apache Meecrowave](https://www.baeldung.com/apache-meecrowave) -- [A Quick Guide to Apache Geode](https://www.baeldung.com/apache-geode) -- [Convert Avro File to JSON File in Java](https://www.baeldung.com/java-avro-json) -- More articles: [[<-- prev]](../apache-libraries) diff --git a/apache-libraries-2/pom.xml b/apache-libraries-2/pom.xml index 28d6582a08ee..f01656cd93a7 100644 --- a/apache-libraries-2/pom.xml +++ b/apache-libraries-2/pom.xml @@ -233,6 +233,16 @@ ${maven.compiler.source} ${maven.compiler.target} + ${maven.compiler.release} + + + + org.apache.maven.plugins + maven-surefire-plugin + + + src/test/resources/logback-test.xml + @@ -257,6 +267,7 @@ 1.15.1 17 17 + 17 \ No newline at end of file diff --git a/apache-libraries-2/src/main/java/com/baeldung/apache/camel/postrequest/PostRequestRoute.java b/apache-libraries-2/src/main/java/com/baeldung/apache/camel/postrequest/PostRequestRoute.java index bf5561caae5d..91901701ebb6 100644 --- a/apache-libraries-2/src/main/java/com/baeldung/apache/camel/postrequest/PostRequestRoute.java +++ b/apache-libraries-2/src/main/java/com/baeldung/apache/camel/postrequest/PostRequestRoute.java @@ -16,9 +16,9 @@ public void configure() throws Exception { .setHeader(Exchange.HTTP_METHOD, constant("POST")) .setHeader(Exchange.CONTENT_TYPE, constant("application/json")) .to("https://jsonplaceholder.typicode.com/posts") - .process(exchange -> log.info("The HTTP response code is: {}", exchange.getIn() + .process(exchange -> log.debug("The HTTP response code is: {}", exchange.getIn() .getHeader(Exchange.HTTP_RESPONSE_CODE))) - .process(exchange -> log.info("The response body is: {}", exchange.getIn() + .process(exchange -> log.debug("The response body is: {}", exchange.getIn() .getBody(String.class))) .to("mock:result"); diff --git a/apache-libraries-2/src/test/resources/logback-test.xml b/apache-libraries-2/src/test/resources/logback-test.xml index 67247b785796..650272f4ac3e 100644 --- a/apache-libraries-2/src/test/resources/logback-test.xml +++ b/apache-libraries-2/src/test/resources/logback-test.xml @@ -13,6 +13,8 @@ + + diff --git a/apache-libraries-3/README.md b/apache-libraries-3/README.md deleted file mode 100644 index c63f3b360bf1..000000000000 --- a/apache-libraries-3/README.md +++ /dev/null @@ -1 +0,0 @@ -## Relevant Articles \ No newline at end of file diff --git a/apache-libraries-3/pom.xml b/apache-libraries-3/pom.xml index 565f59af6528..0e1310a61eed 100644 --- a/apache-libraries-3/pom.xml +++ b/apache-libraries-3/pom.xml @@ -20,12 +20,91 @@ accumulo-core ${accumulo.core} + + org.apache.avro + avro + ${avro.version} + + + net.javacrumbs.json-unit + json-unit-assertj + ${json-unit-assertj.version} + test + + + com.fasterxml.jackson.core + jackson-core + ${jackson.version} + + + com.fasterxml.jackson.dataformat + jackson-dataformat-avro + ${jackson.version} + + + org.apache.camel + camel-jackson + ${camel.version} + + + com.graphql-java + graphql-java + ${graphql.version} + + + org.apache.camel + camel-core + ${camel.version} + + + org.apache.camel + camel-jetty + ${camel.version} + + + org.apache.camel + camel-http + ${camel.version} + + + org.apache.camel + camel-graphql + ${camel.version} + + + org.apache.parquet + parquet-avro + ${parquet.version} + + + org.apache.parquet + parquet-hadoop + ${parquet.version} + + + + + maven-compiler-plugin + ${maven.compiler.plugin} + + ${maven.compiler.source} + + + + + 2.1.3 + 3.14.0 17 17 + 1.11.3 + 3.5.0 + 23.1 + 4.11.0 + 1.16.0 - \ No newline at end of file + diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithAbstractField.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithAbstractField.java new file mode 100644 index 000000000000..e66ce02286ee --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithAbstractField.java @@ -0,0 +1,21 @@ +package com.baeldung.apache.avro.model; + +import org.apache.avro.reflect.Union; + +public record BankAccountWithAbstractField(String bankAccountNumber, + @Union({ PersonalBankAccountReference.class, BusinessBankAccountReference.class }) AccountReference reference) { + + interface AccountReference { + + String reference(); + } + + record PersonalBankAccountReference(String reference, String holderName) implements AccountReference { + + } + + record BusinessBankAccountReference(String reference, String businessEntityId) implements AccountReference { + + } + +} \ No newline at end of file diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithIgnoredField.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithIgnoredField.java new file mode 100644 index 000000000000..9b5da46b9da7 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithIgnoredField.java @@ -0,0 +1,7 @@ +package com.baeldung.apache.avro.model; + +import org.apache.avro.reflect.AvroIgnore; + +public record BankAccountWithIgnoredField(String bankAccountNumber, @AvroIgnore String reference) { + +} \ No newline at end of file diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithLogicalTypes.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithLogicalTypes.java new file mode 100644 index 000000000000..ea584cccf715 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithLogicalTypes.java @@ -0,0 +1,7 @@ +package com.baeldung.apache.avro.model; + +import java.time.LocalDateTime; +import java.util.UUID; + +public record BankAccountWithLogicalTypes(String bankAccountNumber, UUID reference, LocalDateTime expiryDate) { +} \ No newline at end of file diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithNullableField.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithNullableField.java new file mode 100644 index 000000000000..769617455802 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithNullableField.java @@ -0,0 +1,7 @@ +package com.baeldung.apache.avro.model; + +import org.apache.avro.reflect.Nullable; + +public record BankAccountWithNullableField(String bankAccountNumber, @Nullable String reference) { + +} \ No newline at end of file diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithOverriddenField.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithOverriddenField.java new file mode 100644 index 000000000000..311d19e04f08 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/BankAccountWithOverriddenField.java @@ -0,0 +1,7 @@ +package com.baeldung.apache.avro.model; + +import org.apache.avro.reflect.AvroName; + +public record BankAccountWithOverriddenField(String bankAccountNumber, @AvroName("bankAccountReference") String reference) { + +} \ No newline at end of file diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/JacksonBankAccountWithRequiredField.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/JacksonBankAccountWithRequiredField.java new file mode 100644 index 000000000000..887bc1d334e0 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/JacksonBankAccountWithRequiredField.java @@ -0,0 +1,7 @@ +package com.baeldung.apache.avro.model; + +import com.fasterxml.jackson.annotation.JsonProperty; + +public record JacksonBankAccountWithRequiredField(@JsonProperty(required = true) String bankAccountNumber) { + +} \ No newline at end of file diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/SimpleBankAccount.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/SimpleBankAccount.java new file mode 100644 index 000000000000..a4fd040f7e35 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/model/SimpleBankAccount.java @@ -0,0 +1,5 @@ +package com.baeldung.apache.avro.model; + +public record SimpleBankAccount(String bankAccountNumber) { + +} \ No newline at end of file diff --git a/apache-libraries/src/main/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUser.java b/apache-libraries-3/src/main/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUser.java similarity index 100% rename from apache-libraries/src/main/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUser.java rename to apache-libraries-3/src/main/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUser.java diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/camel/Book.java b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/Book.java new file mode 100644 index 000000000000..11e8fec71b19 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/Book.java @@ -0,0 +1,41 @@ +package com.baeldung.apache.camel; + +public class Book { + + private String id; + private String title; + private String author; + + public String getId() { + return id; + } + + public void setId(String id) { + this.id = id; + } + + public String getTitle() { + return title; + } + + public void setTitle(String title) { + this.title = title; + } + + public String getAuthor() { + return author; + } + + public void setAuthor(String author) { + this.author = author; + } + + public Book(String id, String title, String author) { + this.id = id; + this.title = title; + this.author = author; + } + + public Book() { + } +} diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/camel/BookRoute.java b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/BookRoute.java new file mode 100644 index 000000000000..1a4e461913b4 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/BookRoute.java @@ -0,0 +1,95 @@ +package com.baeldung.apache.camel; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.util.Map; + +import org.apache.camel.Exchange; +import org.apache.camel.builder.RouteBuilder; +import org.apache.camel.component.jackson.JacksonDataFormat; +import org.apache.camel.converter.stream.InputStreamCache; +import org.apache.camel.model.dataformat.JsonLibrary; +import org.apache.camel.model.rest.RestBindingMode; +import org.apache.camel.spi.RestConfiguration; + +import com.fasterxml.jackson.core.JsonParseException; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.exc.MismatchedInputException; + +import graphql.ExecutionInput; +import graphql.ExecutionResult; +import graphql.GraphQL; +import graphql.schema.GraphQLSchema; + +public class BookRoute extends RouteBuilder { + + private final BookService bookService = new BookService(); + + @Override + public void configure() throws Exception { + JacksonDataFormat jsonDataFormat = new JacksonDataFormat(Map.class); + jsonDataFormat.setUnmarshalType(Object.class); + + onException(Exception.class).handled(true) + .setHeader("Content-Type", constant("application/json")) + .setBody(simple("{\"error\": \"${exception.message}\"}")); + + onException(IllegalArgumentException.class) + .handled(true) + .setHeader(Exchange.CONTENT_TYPE, constant("application/json")) + .setHeader(Exchange.HTTP_RESPONSE_CODE, constant(400)) + .setBody(simple("{\"error\": \"${exception.message}\"}")); + + onException(JsonParseException.class) + .handled(true) + .setHeader(Exchange.HTTP_RESPONSE_CODE, constant(400)) + .setBody(simple("{\"error\": \"Invalid JSON: ${exception.message}\"}")); + + restConfiguration().component("jetty") + .host("localhost") + .port(8088) + .contextPath("/api") + .bindingMode(RestBindingMode.json) + .dataFormatProperty("prettyPrint", "true"); + + rest("/books") + .get().to("direct:getAllBooks") + .get("/{id}").to("direct:getBookById") + .post() + .consumes("application/json") + .type(Book.class) + .to("direct:addBook"); + + from("direct:getAllBooks").bean(bookService, "getBooks"); + from("direct:getBookById").bean(bookService, "getBookById(${header.id})"); + from("direct:addBook").bean(bookService, "addBook"); + + GraphQLSchema schema = new CustomSchemaLoader().loadSchema(); + GraphQL graphQL = GraphQL.newGraphQL(schema).build(); + + from("jetty:http://localhost:8088/graphql?matchOnUriPrefix=true") + .log("Received GraphQL request: ${body}") + .convertBodyTo(String.class) + .process(exchange -> { + String body = exchange.getIn().getBody(String.class); + try { + Map payload = new ObjectMapper().readValue(body, Map.class); + String query = (String) payload.get("query"); + if (query == null || query.trim().isEmpty()) { + throw new IllegalArgumentException("Missing 'query' field in request body"); + } + ExecutionInput executionInput = ExecutionInput.newExecutionInput() + .query(query) + .build(); + ExecutionResult result = graphQL.execute(executionInput); + Map response = result.toSpecification(); + exchange.getIn().setBody(response); + } catch (Exception e) { + throw new RuntimeException("GraphQL processing error", e); + } + }) + .marshal().json(JsonLibrary.Jackson) + .setHeader(Exchange.CONTENT_TYPE, constant("application/json")); + } +} diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/camel/BookService.java b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/BookService.java new file mode 100644 index 000000000000..bb2f81c47d2a --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/BookService.java @@ -0,0 +1,31 @@ +package com.baeldung.apache.camel; + +import java.util.ArrayList; +import java.util.List; + +public class BookService { + + private final List books = new ArrayList<>(); + + public BookService() { + books.add(new Book("1", "Clean Code", "Robert")); + books.add(new Book("2", "Effective Java", "Joshua")); + } + + public List getBooks() { + return books; + } + + public Book getBookById(String id) { + return books.stream() + .filter(b -> b.getId() + .equals(id)) + .findFirst() + .orElse(null); + } + + public Book addBook(Book book) { + books.add(book); + return book; + } +} diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/camel/CamelRestGraphQLApp.java b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/CamelRestGraphQLApp.java new file mode 100644 index 000000000000..75f77120136e --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/CamelRestGraphQLApp.java @@ -0,0 +1,21 @@ +package com.baeldung.apache.camel; + +import org.apache.camel.CamelContext; +import org.apache.camel.impl.DefaultCamelContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import graphql.GraphQL; +import graphql.schema.GraphQLSchema; + +public class CamelRestGraphQLApp { + private static final Logger logger = LoggerFactory.getLogger(CamelRestGraphQLApp.class); + public static void main(String[] args) throws Exception { + CamelContext context = new DefaultCamelContext(); + context.addRoutes(new BookRoute()); + context.start(); + logger.info("Server running at http://localhost:8088"); + Thread.sleep(Long.MAX_VALUE); + context.stop(); + } +} diff --git a/apache-libraries-3/src/main/java/com/baeldung/apache/camel/CustomSchemaLoader.java b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/CustomSchemaLoader.java new file mode 100644 index 000000000000..9152d3a16011 --- /dev/null +++ b/apache-libraries-3/src/main/java/com/baeldung/apache/camel/CustomSchemaLoader.java @@ -0,0 +1,55 @@ +package com.baeldung.apache.camel; + +import java.io.InputStream; +import java.io.InputStreamReader; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import graphql.schema.GraphQLSchema; +import graphql.schema.idl.RuntimeWiring; +import graphql.schema.idl.SchemaGenerator; +import graphql.schema.idl.SchemaParser; +import graphql.schema.idl.TypeDefinitionRegistry; + +public class CustomSchemaLoader { + + private static final Logger logger = LoggerFactory.getLogger(CustomSchemaLoader.class); + private final BookService bookService = new BookService(); + + public GraphQLSchema loadSchema() { + logger.debug("Attempting to load schema"); + try (InputStream schemaStream = getClass().getClassLoader().getResourceAsStream("books.graphql")) { + if (schemaStream == null) { + throw new RuntimeException("GraphQL schema file 'books.graphql' not found in classpath"); + } + logger.debug("Schema file found. Parsing..."); + TypeDefinitionRegistry registry = new SchemaParser() + .parse(new InputStreamReader(schemaStream)); + + RuntimeWiring wiring = buildRuntimeWiring(); + + return new SchemaGenerator().makeExecutableSchema(registry, wiring); + + } catch (Exception e) { + logger.error("Failed to load GraphQL schema", e); + throw new RuntimeException("GraphQL schema initialization failed", e); + } + } + + private RuntimeWiring buildRuntimeWiring() { + return RuntimeWiring.newRuntimeWiring() + .type("Query", builder -> builder + .dataFetcher("books", env -> bookService.getBooks()) + .dataFetcher("bookById", env -> bookService.getBookById(env.getArgument("id")))) + .type("Mutation", builder -> builder + .dataFetcher("addBook", env -> { + String id = env.getArgument("id"); + String title = env.getArgument("title"); + String author = env.getArgument("author"); + if (title == null || title.isEmpty()) { + throw new IllegalArgumentException("Title cannot be empty"); + } + return bookService.addBook(new Book(id, title, author)); + })) + .build(); + } +} diff --git a/apache-libraries-3/src/main/resources/books.graphql b/apache-libraries-3/src/main/resources/books.graphql new file mode 100644 index 000000000000..fcc0d801936b --- /dev/null +++ b/apache-libraries-3/src/main/resources/books.graphql @@ -0,0 +1,14 @@ +type Book { + id: String! + title: String! + author: String +} + +type Query { + books: [Book] + bookById(id: String!): Book +} + +type Mutation { + addBook(id: String!, title: String!, author: String): Book +} \ No newline at end of file diff --git a/apache-libraries-3/src/test/java/com/baeldung/apache/avro/JacksonAvroUnitTest.java b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/JacksonAvroUnitTest.java new file mode 100644 index 000000000000..197a2fad0feb --- /dev/null +++ b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/JacksonAvroUnitTest.java @@ -0,0 +1,105 @@ +package com.baeldung.apache.avro; + +import static net.javacrumbs.jsonunit.assertj.JsonAssertions.assertThatJson; + +import org.apache.avro.Schema; +import org.junit.jupiter.api.Test; + +import com.baeldung.apache.avro.model.BankAccountWithLogicalTypes; +import com.baeldung.apache.avro.model.JacksonBankAccountWithRequiredField; +import com.baeldung.apache.avro.model.SimpleBankAccount; +import com.fasterxml.jackson.databind.JsonMappingException; +import com.fasterxml.jackson.dataformat.avro.AvroMapper; +import com.fasterxml.jackson.dataformat.avro.jsr310.AvroJavaTimeModule; +import com.fasterxml.jackson.dataformat.avro.schema.AvroSchemaGenerator; + +class JacksonAvroUnitTest { + + @Test + void whenConvertingRecord_thenAvroSchemaIsCorrect() throws JsonMappingException { + AvroMapper avroMapper = new AvroMapper(); + AvroSchemaGenerator avroSchemaGenerator = new AvroSchemaGenerator(); + + avroMapper.acceptJsonFormatVisitor(SimpleBankAccount.class, avroSchemaGenerator); + Schema schema = avroSchemaGenerator.getGeneratedSchema() + .getAvroSchema(); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "SimpleBankAccount", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : [ "null", "string" ] + } ] + } + """); + } + + @Test + void whenConvertingRecordWithRequiredField_thenAvroSchemaIsCorrect() throws JsonMappingException { + AvroMapper avroMapper = new AvroMapper(); + AvroSchemaGenerator avroSchemaGenerator = new AvroSchemaGenerator(); + + avroMapper.acceptJsonFormatVisitor(JacksonBankAccountWithRequiredField.class, avroSchemaGenerator); + Schema schema = avroSchemaGenerator.getGeneratedSchema() + .getAvroSchema(); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "JacksonBankAccountWithRequiredField", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : "string" + } ] + } + """); + } + + @Test + void whenConvertingRecordWithLogicalField_whenGeneratingSchemaWithJackson_thenAvroSchemaIsCorrect() throws JsonMappingException { + AvroMapper avroMapper = AvroMapper.builder() + .addModule(new AvroJavaTimeModule()) + .build(); + + AvroSchemaGenerator avroSchemaGenerator = new AvroSchemaGenerator().enableLogicalTypes(); + + avroMapper.acceptJsonFormatVisitor(BankAccountWithLogicalTypes.class, avroSchemaGenerator); + Schema schema = avroSchemaGenerator.getGeneratedSchema() + .getAvroSchema(); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "BankAccountWithLogicalTypes", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : [ "null", "string" ] + }, { + "name" : "expiryDate", + "type" : [ "null", { + "type" : "long", + "logicalType" : "local-timestamp-millis" + } ] + }, { + "name" : "reference", + "type" : [ "null", { + "type" : "fixed", + "name" : "UUID", + "namespace" : "java.util", + "doc" : "", + "size" : 16 + } ] + } ] + } + """); + } + +} diff --git a/apache-libraries-3/src/test/java/com/baeldung/apache/avro/ReflectDataUnitTest.java b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/ReflectDataUnitTest.java new file mode 100644 index 000000000000..7c71c47ddd64 --- /dev/null +++ b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/ReflectDataUnitTest.java @@ -0,0 +1,181 @@ +package com.baeldung.apache.avro; + +import static net.javacrumbs.jsonunit.assertj.JsonAssertions.assertThatJson; + +import org.apache.avro.Conversions; +import org.apache.avro.Schema; +import org.apache.avro.data.TimeConversions; +import org.apache.avro.reflect.ReflectData; +import org.junit.jupiter.api.Test; + +import com.baeldung.apache.avro.model.BankAccountWithAbstractField; +import com.baeldung.apache.avro.model.BankAccountWithIgnoredField; +import com.baeldung.apache.avro.model.BankAccountWithLogicalTypes; +import com.baeldung.apache.avro.model.BankAccountWithNullableField; +import com.baeldung.apache.avro.model.BankAccountWithOverriddenField; +import com.baeldung.apache.avro.model.SimpleBankAccount; + +class ReflectDataUnitTest { + + @Test + void whenConvertingSimpleRecord_thenAvroSchemaIsCorrect() { + Schema schema = ReflectData.get() + .getSchema(SimpleBankAccount.class); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "SimpleBankAccount", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : "string" + } ] + } + """); + } + + @Test + void whenConvertingRecordWithNullableField_thenAvroSchemaIsCorrect() { + Schema schema = ReflectData.get() + .getSchema(BankAccountWithNullableField.class); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "BankAccountWithNullableField", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : "string" + }, { + "name" : "reference", + "type" : [ "null", "string" ], + "default" : null + } ] + } + """); + } + + @Test + void whenConvertingRecordWithIgnoredField_thenAvroSchemaIsCorrect() { + Schema schema = ReflectData.get() + .getSchema(BankAccountWithIgnoredField.class); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "BankAccountWithIgnoredField", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : "string" + } ] + } + """); + } + + @Test + void whenConvertingRecordWithOverriddenField_thenAvroSchemaIsCorrect() { + Schema schema = ReflectData.get() + .getSchema(BankAccountWithOverriddenField.class); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "BankAccountWithOverriddenField", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : "string" + }, { + "name" : "bankAccountReference", + "type" : "string" + } ] + } + """); + } + + @Test + void whenConvertingRecordWithAbstractField_thenAvroSchemaIsCorrect() { + Schema schema = ReflectData.get() + .getSchema(BankAccountWithAbstractField.class); + String jsonSchema = schema.toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "BankAccountWithAbstractField", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : "string" + }, { + "name" : "reference", + "type" : [ { + "type" : "record", + "name" : "PersonalBankAccountReference", + "namespace" : "com.baeldung.apache.avro.model.BankAccountWithAbstractField", + "fields" : [ { + "name" : "holderName", + "type" : "string" + }, { + "name" : "reference", + "type" : "string" + } ] + }, { + "type" : "record", + "name" : "BusinessBankAccountReference", + "namespace" : "com.baeldung.apache.avro.model.BankAccountWithAbstractField", + "fields" : [ { + "name" : "businessEntityId", + "type" : "string" + }, { + "name" : "reference", + "type" : "string" + } ] + } ] + } ] + } + """); + } + + @Test + void whenConvertingRecordWithLogicalTypes_thenAvroSchemaIsCorrect() { + ReflectData reflectData = ReflectData.get(); + reflectData.addLogicalTypeConversion(new Conversions.UUIDConversion()); + reflectData.addLogicalTypeConversion(new TimeConversions.LocalTimestampMillisConversion()); + + String jsonSchema = reflectData.getSchema(BankAccountWithLogicalTypes.class) + .toString(); + + assertThatJson(jsonSchema).isEqualTo(""" + { + "type" : "record", + "name" : "BankAccountWithLogicalTypes", + "namespace" : "com.baeldung.apache.avro.model", + "fields" : [ { + "name" : "bankAccountNumber", + "type" : "string" + }, { + "name" : "expiryDate", + "type" : { + "type" : "long", + "logicalType" : "local-timestamp-millis" + } + }, { + "name" : "reference", + "type" : { + "type" : "string", + "logicalType" : "uuid" + } + } ] + } + """); + } + +} \ No newline at end of file diff --git a/apache-libraries-3/src/test/java/com/baeldung/apache/avro/SerializeAndDeserializeDateUnitTest.java b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/SerializeAndDeserializeDateUnitTest.java new file mode 100644 index 000000000000..ad36cbd1ff23 --- /dev/null +++ b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/SerializeAndDeserializeDateUnitTest.java @@ -0,0 +1,195 @@ +package com.baeldung.apache.avro; + +import org.apache.avro.Conversion; +import org.apache.avro.LogicalTypes; +import org.apache.avro.Schema; +import org.apache.avro.data.TimeConversions; +import org.apache.avro.generic.GenericData; +import org.apache.avro.generic.GenericDatumReader; +import org.apache.avro.generic.GenericDatumWriter; +import org.apache.avro.generic.GenericRecord; +import org.apache.avro.io.DatumReader; +import org.apache.avro.io.DatumWriter; +import org.apache.avro.io.Decoder; +import org.apache.avro.io.DecoderFactory; +import org.apache.avro.io.Encoder; +import org.apache.avro.io.EncoderFactory; +import org.apache.commons.lang3.tuple.Pair; +import org.junit.jupiter.api.Test; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.time.Instant; +import java.time.LocalDate; +import java.time.ZoneId; +import java.util.Date; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +public class SerializeAndDeserializeDateUnitTest { + + @Test + void whenSerializingDateWithLogicalType_thenDeserializesCorrectly() throws IOException { + + LocalDate expectedDate = LocalDate.now(); + Instant expectedTimestamp = Instant.now(); + + byte[] serialized = serializeDateWithLogicalType(expectedDate, expectedTimestamp); + Pair deserialized = deserializeDateWithLogicalType(serialized); + + assertEquals(expectedDate, deserialized.getLeft()); + + // This is perfectly valid when using logical types + assertEquals(expectedTimestamp.toEpochMilli(), deserialized.getRight().toEpochMilli(), + "Timestamps should match exactly at millisecond precision"); + } + + @Test + void whenSerializingWithConversionApi_thenDeserializesCorrectly() throws IOException { + + LocalDate expectedDate = LocalDate.now(); + Instant expectedTimestamp = Instant.now(); + + byte[] serialized = serializeWithConversionApi(expectedDate, expectedTimestamp); + Pair deserialized = deserializeWithConversionApi(serialized); + + assertEquals(expectedDate, deserialized.getLeft()); + assertEquals(expectedTimestamp.toEpochMilli(), deserialized.getRight().toEpochMilli(), + "Timestamps should match at millisecond precision"); + } + + @Test + void whenSerializingLegacyDate_thenConvertsCorrectly() throws IOException { + + Date legacyDate = new Date(); + LocalDate expectedLocalDate = legacyDate.toInstant() + .atZone(ZoneId.systemDefault()) + .toLocalDate(); + + byte[] serialized = serializeLegacyDateAsModern(legacyDate); + LocalDate deserialized = deserializeDateWithLogicalType(serialized).getKey(); + + assertEquals(expectedLocalDate, deserialized); + } + + public static Schema createDateSchema() { + String schemaJson = + "{" + + "\"type\": \"record\"," + + "\"name\": \"DateRecord\"," + + "\"fields\": [" + + " {\"name\": \"date\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}," + + " {\"name\": \"timestamp\", \"type\": {\"type\": \"long\", \"logicalType\": \"timestamp-millis\"}}" + + "]" + + "}"; + return new Schema.Parser().parse(schemaJson); + } + + public static byte[] serializeDateWithLogicalType(LocalDate date, Instant timestamp) throws IOException { + Schema schema = createDateSchema(); + GenericRecord record = new GenericData.Record(schema); + + // Convert LocalDate to days since epoch + record.put("date", (int) date.toEpochDay()); + + // Convert Instant to milliseconds since epoch + record.put("timestamp", timestamp.toEpochMilli()); + + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + DatumWriter datumWriter = new GenericDatumWriter<>(schema); + Encoder encoder = EncoderFactory.get().binaryEncoder(baos, null); + + datumWriter.write(record, encoder); + encoder.flush(); + + return baos.toByteArray(); + } + + public static Pair deserializeDateWithLogicalType(byte[] bytes) throws IOException { + Schema schema = createDateSchema(); + DatumReader datumReader = new GenericDatumReader<>(schema); + Decoder decoder = DecoderFactory.get().binaryDecoder(bytes, null); + + GenericRecord record = datumReader.read(null, decoder); + + // Convert days since epoch back to LocalDate + LocalDate date = LocalDate.ofEpochDay((int) record.get("date")); + + // Convert milliseconds since epoch back to Instant + Instant timestamp = Instant.ofEpochMilli((long) record.get("timestamp")); + + return Pair.of(date, timestamp); + } + + public static byte[] serializeWithConversionApi(LocalDate date, Instant timestamp) throws IOException { + Schema schema = createDateSchema(); + GenericRecord record = new GenericData.Record(schema); + + // Use LogicalTypes.date() for conversion + Conversion dateConversion = new org.apache.avro.data.TimeConversions.DateConversion(); + LogicalTypes.date().addToSchema(schema.getField("date").schema()); + + // Use LogicalTypes.timestampMillis() for conversion + Conversion timestampConversion = new org.apache.avro.data.TimeConversions.TimestampMillisConversion(); + LogicalTypes.timestampMillis().addToSchema(schema.getField("timestamp").schema()); + + record.put("date", dateConversion.toInt(date, schema.getField("date").schema(), LogicalTypes.date())); + record.put("timestamp", timestampConversion.toLong(timestamp, schema.getField("timestamp").schema(), LogicalTypes.timestampMillis())); + + // Serialize as before + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + DatumWriter datumWriter = new GenericDatumWriter<>(schema); + Encoder encoder = EncoderFactory.get().binaryEncoder(baos, null); + + datumWriter.write(record, encoder); + encoder.flush(); + + return baos.toByteArray(); + } + + public static Pair deserializeWithConversionApi(byte[] bytes) throws IOException { + Schema schema = createDateSchema(); + DatumReader datumReader = new GenericDatumReader<>(schema); + Decoder decoder = DecoderFactory.get().binaryDecoder(bytes, null); + + GenericRecord record = datumReader.read(null, decoder); + + // Use LogicalTypes.date() for conversion + Conversion dateConversion = new TimeConversions.DateConversion(); + LogicalTypes.date().addToSchema(schema.getField("date").schema()); + + // Use LogicalTypes.timestampMillis() for conversion + Conversion timestampConversion = new TimeConversions.TimestampMillisConversion(); + LogicalTypes.timestampMillis().addToSchema(schema.getField("timestamp").schema()); + + // Get the primitive values from the record + int daysSinceEpoch = (int) record.get("date"); + long millisSinceEpoch = (long) record.get("timestamp"); + + // Convert back to Java types using the conversion API + LocalDate date = dateConversion.fromInt( + daysSinceEpoch, + schema.getField("date").schema(), + LogicalTypes.date() + ); + + Instant timestamp = timestampConversion.fromLong( + millisSinceEpoch, + schema.getField("timestamp").schema(), + LogicalTypes.timestampMillis() + ); + + return Pair.of(date, timestamp); + } + + public static byte[] serializeLegacyDateAsModern(Date legacyDate) throws IOException { + // Convert java.util.Date to java.time.Instant + Instant instant = legacyDate.toInstant(); + + // Convert to LocalDate if you need date-only information + LocalDate localDate = instant.atZone(ZoneId.systemDefault()).toLocalDate(); + + // Then use one of our modern date serialization methods + return serializeDateWithLogicalType(localDate, instant); + } +} diff --git a/apache-libraries-3/src/test/java/com/baeldung/apache/avro/SerializeEnumValueInAvroUnitTest.java b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/SerializeEnumValueInAvroUnitTest.java new file mode 100644 index 000000000000..9d8e84ea887b --- /dev/null +++ b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/SerializeEnumValueInAvroUnitTest.java @@ -0,0 +1,179 @@ +package com.baeldung.apache.avro; + +import org.apache.avro.Schema; +import org.apache.avro.SchemaBuilder; +import org.apache.avro.file.DataFileReader; +import org.apache.avro.file.DataFileWriter; +import org.apache.avro.generic.GenericData; +import org.apache.avro.generic.GenericDatumReader; +import org.apache.avro.generic.GenericDatumWriter; +import org.apache.avro.generic.GenericRecord; +import org.apache.avro.io.DatumReader; +import org.apache.avro.io.DatumWriter; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.io.File; +import java.io.IOException; +import java.nio.file.Path; + +import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; +import static org.junit.jupiter.api.Assertions.assertEquals; + +public class SerializeEnumValueInAvroUnitTest { + + @TempDir + Path tempDir; + + private Schema colorEnum; + private Schema recordSchema; + private Schema unionSchema; + private Schema recordWithUnionSchema; + + @BeforeEach + void setUp() { + // Create enum schema + colorEnum = SchemaBuilder.enumeration("Color") + .namespace("com.baeldung.apache.avro") + .symbols("UNKNOWN", "GREEN", "RED", "BLUE"); + + // Create record schema with enum field + recordSchema = SchemaBuilder.record("ColorRecord") + .namespace("com.baeldung.apache.avro") + .fields() + .name("color") + .type(colorEnum) + .noDefault() + .endRecord(); + + // Create union schema with enum and null + unionSchema = SchemaBuilder.unionOf() + .type(colorEnum) + .and() + .nullType() + .endUnion(); + + // Create record schema with union field + recordWithUnionSchema = SchemaBuilder.record("ColorRecordWithUnion") + .namespace("com.baeldung.apache.avro") + .fields() + .name("color") + .type(unionSchema) + .noDefault() + .endRecord(); + } + + @Test + void whenSerializingEnum_thenSuccess() throws IOException { + File file = tempDir.resolve("color.avro").toFile(); + + // Create record with enum value + GenericRecord record = new GenericData.Record(recordSchema); + GenericData.EnumSymbol colorSymbol = new GenericData.EnumSymbol(colorEnum, "RED"); + record.put("color", colorSymbol); + + // Write to file + DatumWriter datumWriter = new GenericDatumWriter<>(recordSchema); + try (DataFileWriter dataFileWriter = new DataFileWriter<>(datumWriter)) { + dataFileWriter.create(recordSchema, file); + dataFileWriter.append(record); + } + + // Read from file + DatumReader datumReader = new GenericDatumReader<>(recordSchema); + try (DataFileReader dataFileReader = new DataFileReader<>(file, datumReader)) { + GenericRecord result = dataFileReader.next(); + assertEquals("RED", result.get("color").toString()); + } + } + + @Test + void whenSerializingEnumInUnion_thenSuccess() throws IOException { + File file = tempDir.resolve("colorUnion.avro").toFile(); + + // Create record with enum in union + GenericRecord record = new GenericData.Record(recordWithUnionSchema); + GenericData.EnumSymbol colorSymbol = new GenericData.EnumSymbol(colorEnum, "GREEN"); + record.put("color", colorSymbol); + + // Write to file + DatumWriter datumWriter = new GenericDatumWriter<>(recordWithUnionSchema); + try (DataFileWriter dataFileWriter = new DataFileWriter<>(datumWriter)) { + dataFileWriter.create(recordWithUnionSchema, file); + dataFileWriter.append(record); + } + + // Read from file + DatumReader datumReader = new GenericDatumReader<>(recordWithUnionSchema); + try (DataFileReader dataFileReader = new DataFileReader<>(file, datumReader)) { + GenericRecord result = dataFileReader.next(); + assertEquals("GREEN", result.get("color").toString()); + } + } + + @Test + void whenSerializingNullInUnion_thenSuccess() throws IOException { + File file = tempDir.resolve("colorNull.avro").toFile(); + + // Create record with null in union + GenericRecord record = new GenericData.Record(recordWithUnionSchema); + record.put("color", null); + + // Write to file + DatumWriter datumWriter = new GenericDatumWriter<>(recordWithUnionSchema); + assertDoesNotThrow(() -> { + try (DataFileWriter dataFileWriter = new DataFileWriter<>(datumWriter)) { + dataFileWriter.create(recordWithUnionSchema, file); + dataFileWriter.append(record); + } + }); + } + + @Test + void whenSchemaEvolution_thenDefaultValueUsed() throws IOException { + // Create schema with new enum value and default at schema level + String evolvedSchemaJson = "{\"type\":\"record\"," + + "\"name\":\"ColorRecord\"," + + "\"namespace\":\"com.baeldung.apache.avro\"," + + "\"fields\":[{\"name\":\"color\"," + + "\"type\":{\"type\":\"enum\"," + + "\"name\":\"Color\"," + + "\"symbols\":[\"UNKNOWN\",\"GREEN\",\"RED\",\"BLUE\",\"YELLOW\"]," + + "\"default\":\"UNKNOWN\"}}]}"; + + Schema evolvedRecordSchema = new Schema.Parser().parse(evolvedSchemaJson); + Schema evolvedEnum = evolvedRecordSchema.getField("color").schema(); + + File file = tempDir.resolve("colorEvolved.avro").toFile(); + + // Create record with new enum value + GenericRecord record = new GenericData.Record(evolvedRecordSchema); + GenericData.EnumSymbol colorSymbol = new GenericData.EnumSymbol(evolvedEnum, "YELLOW"); + record.put("color", colorSymbol); + + // Write with evolved schema + DatumWriter datumWriter = new GenericDatumWriter<>(evolvedRecordSchema); + try (DataFileWriter dataFileWriter = new DataFileWriter<>(datumWriter)) { + dataFileWriter.create(evolvedRecordSchema, file); + dataFileWriter.append(record); + } + + // Create old schema without YELLOW but WITH default + String originalSchemaJson = "{\"type\":\"record\"," + + "\"name\":\"ColorRecord\"," + + "\"namespace\":\"com.baeldung.apache.avro\"," + + "\"fields\":[{\"name\":\"color\",\"type\":{\"type\":\"enum\",\"name\":\"Color\"," + + "\"symbols\":[\"UNKNOWN\",\"GREEN\",\"RED\",\"BLUE\"]," + + "\"default\":\"UNKNOWN\"}}]}"; + + Schema originalRecordSchema = new Schema.Parser().parse(originalSchemaJson); + + // Read with original schema + DatumReader datumReader = new GenericDatumReader<>(evolvedRecordSchema, originalRecordSchema); + try (DataFileReader dataFileReader = new DataFileReader<>(file, datumReader)) { + GenericRecord result = dataFileReader.next(); + assertEquals("UNKNOWN", result.get("color").toString()); + } + } +} diff --git a/apache-libraries/src/test/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUserUnitTest.java b/apache-libraries-3/src/test/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUserUnitTest.java similarity index 100% rename from apache-libraries/src/test/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUserUnitTest.java rename to apache-libraries-3/src/test/java/com/baeldung/apache/avro/storingnullvaluesinavrofile/AvroUserUnitTest.java diff --git a/apache-libraries-3/src/test/java/com/baeldung/apache/camel/CamelRestGraphQLAppUnitTest.java b/apache-libraries-3/src/test/java/com/baeldung/apache/camel/CamelRestGraphQLAppUnitTest.java new file mode 100644 index 000000000000..8e90adaea6e1 --- /dev/null +++ b/apache-libraries-3/src/test/java/com/baeldung/apache/camel/CamelRestGraphQLAppUnitTest.java @@ -0,0 +1,165 @@ +package com.baeldung.apache.camel; +import org.apache.camel.CamelContext; +import org.apache.camel.Exchange; +import org.apache.camel.ProducerTemplate; +import org.apache.camel.impl.DefaultCamelContext; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.*; + + +public class CamelRestGraphQLAppUnitTest { + + private static CamelContext context; + private static ProducerTemplate template; + + @BeforeAll + public static void setup() throws Exception { + context = new DefaultCamelContext(); + context.addRoutes(new BookRoute()); + context.start(); + template = context.createProducerTemplate(); + Thread.sleep(2000); + } + + @AfterAll + public static void tearDown() throws Exception { + if (template != null) { + template.stop(); + } + if (context != null) { + context.stop(); + } + } + + @Test + void whenCallingRestGetAllBooks_thenReturnBookList() { + String response = template.requestBodyAndHeader( + "http://localhost:8088/api/books", + null, + Exchange.CONTENT_TYPE, + "application/json", + String.class + ); + + assertNotNull(response); + assertTrue(response.contains("Clean Code")); + assertTrue(response.contains("Effective Java")); + assertTrue(response.contains("Robert")); + assertTrue(response.contains("Joshua")); + } + + @Test + void whenCallingRestGetBookById_thenReturnSpecificBook() { + String response = template.requestBody("http://localhost:8088/api/books/1", null, String.class); + + assertNotNull(response); + assertTrue(response.contains("Clean Code")); + assertTrue(response.contains("Robert")); + assertFalse(response.contains("Effective Java")); + } + + @Test + void whenPostingNewBook_thenAddToCollection() { + String bookJson = "{\"id\":\"3\",\"title\":\"Camel in Action\",\"author\":\"Claus Ibsen\"}"; + + String postResponse = template.requestBodyAndHeader( + "http://localhost:8088/api/books", + bookJson, + "Content-Type", + "application/json", + String.class + ); + + String getResponse = template.requestBody("http://localhost:8088/api/books", null, String.class); + + assertNotNull(postResponse); + assertTrue(getResponse.contains("Camel in Action")); + assertTrue(getResponse.contains("Claus Ibsen")); + } + + @Test + void whenCallingBooksQuery_thenReturnAllBooks() { + String query = """ + { + "query": "{ books { id title author } }" + }"""; + + String response = template.requestBodyAndHeader( + "http://localhost:8088/graphql", + query, + Exchange.CONTENT_TYPE, + "application/json", + String.class + ); + assertNotNull(response); + + assertTrue(response.contains("books")); + assertTrue(response.contains("Clean Code")); + assertTrue(response.contains("Effective Java")); + assertTrue(response.contains("Robert")); + assertTrue(response.contains("Joshua")); + } + + @Test + void whenCallingBookByIdQuery_thenReturnSpecificBook() { + String query = "{\"query\":\"{ bookById(id: \\\"1\\\") { title author } }\"}"; + + String response = template.requestBodyAndHeader( + "http://localhost:8088/graphql", + query, + "Content-Type", + "application/json", + String.class + ); + + assertNotNull(response); + assertTrue(response.contains("Clean Code")); + assertTrue(response.contains("Robert")); + assertFalse(response.contains("Effective Java")); + } + + @Test + void whenAddingBookViaMutation_thenPersist() { + String bookJson = "{ \"id\": \"3\", \"title\": \"Camel in Action\", \"author\": \"Claus Ibsen\" }"; + + String postResponse = template.requestBodyAndHeader( + "http://localhost:8088/api/books", + bookJson, + Exchange.CONTENT_TYPE, + "application/json", + String.class + ); + + String queryResponse = template.requestBodyAndHeader( + "http://localhost:8088/graphql", + "{\"query\":\"{ books { title } }\"}", + "Content-Type", + "application/json", + String.class + ); + + assertNotNull(postResponse); + assertTrue(postResponse.contains("Camel in Action")); + } + + @Test + void whenAddingInvalidBook_thenReturnError() { + String mutation = "{\"query\":\"mutation { " + + "addBook(id: \\\"4\\\", title: \\\"\\\", author: \\\"Test\\\") { id title }" + + "}\"}"; + + String response = template.requestBodyAndHeader( + "http://localhost:8088/graphql", + mutation, + "Content-Type", + "application/json", + String.class + ); + + assertNotNull(response); + assertTrue(response.contains("errors")); + assertTrue(response.contains("Title cannot be empty")); + } +} \ No newline at end of file diff --git a/apache-libraries-3/src/test/java/com/baeldung/apache/parquet/ParquetJavaUnitTest.java b/apache-libraries-3/src/test/java/com/baeldung/apache/parquet/ParquetJavaUnitTest.java new file mode 100644 index 000000000000..67d251932af7 --- /dev/null +++ b/apache-libraries-3/src/test/java/com/baeldung/apache/parquet/ParquetJavaUnitTest.java @@ -0,0 +1,358 @@ +package com.baeldung.apache.parquet; + +import org.apache.avro.Schema; +import org.apache.avro.generic.GenericData; +import org.apache.avro.generic.GenericRecord; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.Path; +import org.apache.parquet.avro.AvroParquetReader; +import org.apache.parquet.avro.AvroParquetWriter; +import org.apache.parquet.avro.AvroReadSupport; +import org.apache.parquet.column.Encoding; +import org.apache.parquet.column.EncodingStats; +import org.apache.parquet.column.ParquetProperties; +import org.apache.parquet.example.data.Group; +import org.apache.parquet.example.data.simple.SimpleGroupFactory; +import org.apache.parquet.filter2.compat.FilterCompat; +import org.apache.parquet.filter2.predicate.FilterApi; +import org.apache.parquet.filter2.predicate.FilterPredicate; +import org.apache.parquet.hadoop.ParquetFileReader; +import org.apache.parquet.hadoop.ParquetReader; +import org.apache.parquet.hadoop.ParquetWriter; +import org.apache.parquet.hadoop.example.ExampleParquetWriter; +import org.apache.parquet.hadoop.example.GroupReadSupport; +import org.apache.parquet.hadoop.example.GroupWriteSupport; +import org.apache.parquet.hadoop.metadata.BlockMetaData; +import org.apache.parquet.hadoop.metadata.ColumnChunkMetaData; +import org.apache.parquet.hadoop.metadata.CompressionCodecName; +import org.apache.parquet.hadoop.metadata.ParquetMetadata; +import org.apache.parquet.hadoop.util.HadoopInputFile; +import org.apache.parquet.hadoop.util.HadoopOutputFile; +import org.apache.parquet.io.InputFile; +import org.apache.parquet.io.OutputFile; +import org.apache.parquet.schema.LogicalTypeAnnotation; +import org.apache.parquet.schema.MessageType; +import org.apache.parquet.schema.MessageTypeParser; +import org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName; +import org.apache.parquet.schema.Types; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.List; +import java.util.Set; +import java.util.logging.Logger; + +import static org.junit.jupiter.api.Assertions.*; + +public class ParquetJavaUnitTest { + + @Nested + class AvroUnitTest { + + private static final String PERSON_AVRO = """ + { + "type":"record", + "name":"Person", + "namespace":"com.baeldung.avro", + "fields":[ + {"name":"name","type":"string"}, + {"name":"age","type":"int"}, + {"name":"city","type":["null","string"],"default":null} + ] + } + """; + + private static final String NAME_ONLY = """ + { + "type":"record", + "name":"OnlyName", + "fields":[ + { + "name":"name", + "type":"string" + } + ] + } + """; + + @Test + void givenAvroSchema_whenWritingAndReadingWithAvroParquet_thenFirstRecordMatches(@TempDir java.nio.file.Path tmp) throws Exception { + Schema schema = new Schema.Parser().parse(PERSON_AVRO); + Configuration conf = new Configuration(); + Path hPath = new Path(tmp.resolve("people-avro.parquet").toUri()); + OutputFile out = HadoopOutputFile.fromPath(hPath, conf); + + try (ParquetWriter writer = AvroParquetWriter. builder(out) + .withSchema(schema) + .withConf(conf) + .build()) { + GenericRecord r1 = new GenericData.Record(schema); + r1.put("name", "Carla"); + r1.put("age", 41); + r1.put("city", "Milan"); + + GenericRecord r2 = new GenericData.Record(schema); + r2.put("name", "Diego"); + r2.put("age", 23); + r2.put("city", null); + + writer.write(r1); + writer.write(r2); + } + + InputFile in = HadoopInputFile.fromPath(hPath, conf); + + try (ParquetReader reader = AvroParquetReader. builder(in) + .withConf(conf) + .build()) { + GenericRecord first = reader.read(); + assertEquals("Carla", first.get("name").toString()); + assertEquals(41, first.get("age")); + } + } + + @Test + void givenProjectionSchema_whenReading_thenNonProjectedFieldsAreNull(@TempDir java.nio.file.Path tmp) throws Exception { + Configuration conf = new Configuration(); + + Schema writeSchema = new Schema.Parser().parse(PERSON_AVRO); + Path hPath = new Path(tmp.resolve("people-avro.parquet").toUri()); + + try (ParquetWriter writer = AvroParquetWriter. builder(HadoopOutputFile.fromPath(hPath, conf)) + .withSchema(writeSchema) + .withConf(conf) + .build()) { + GenericRecord r = new GenericData.Record(writeSchema); + r.put("name", "Alice"); + r.put("age", 30); + r.put("city", null); + writer.write(r); + } + + Schema projection = new Schema.Parser().parse(NAME_ONLY); + AvroReadSupport.setRequestedProjection(conf, projection); + + InputFile in = HadoopInputFile.fromPath(hPath, conf); + try (ParquetReader reader = AvroParquetReader. builder(in) + .withConf(conf) + .build()) { + GenericRecord rec = reader.read(); + assertNotNull(rec.get("name")); + assertNull(rec.get("age")); + } + } + } + + @Nested + class ExampleApiUnitTest { + + @Test + void givenSchema_whenWritingAndReadingWithExampleApi_thenRoundtripWorks(@TempDir java.nio.file.Path tmp) throws Exception { + String schemaString = """ + message person { + required binary name (UTF8); + required int32 age; + optional binary city (UTF8); + } + """; + MessageType schema = MessageTypeParser.parseMessageType(schemaString); + SimpleGroupFactory factory = new SimpleGroupFactory(schema); + Configuration conf = new Configuration(); + Path hPath = new Path(tmp.resolve("people-example.parquet").toUri()); + + try (ParquetWriter writer = ExampleParquetWriter.builder(HadoopOutputFile.fromPath(hPath, conf)) + .withConf(conf) + .withType(schema) + .build()) { + writer.write(factory.newGroup() + .append("name", "Alice") + .append("age", 34) + .append("city", "Rome")); + writer.write(factory.newGroup() + .append("name", "Bob") + .append("age", 29)); + } + + List names = new ArrayList<>(); + try (ParquetReader reader = ParquetReader.builder(new GroupReadSupport(), hPath) + .withConf(conf) + .build()) { + Group g; + while ((g = reader.read()) != null) { + names.add(g.getBinary("name", 0).toStringUsingUTF8()); + } + } + assertEquals(List.of("Alice", "Bob"), names); + } + + @Test + void givenAgeFilter_whenReading_thenOnlyMatchingRowsAppear(@TempDir java.nio.file.Path tmp) throws Exception { + Configuration conf = new Configuration(); + + MessageType schema = Types.buildMessage() + .addField(Types.required(PrimitiveTypeName.BINARY) + .as(LogicalTypeAnnotation.stringType()) + .named("name")) + .addField(Types.required(PrimitiveTypeName.INT32) + .named("age")) + .named("Person"); + + GroupWriteSupport.setSchema(schema, conf); + Path hPath = new Path(tmp.resolve("people-example.parquet").toUri()); + + try (ParquetWriter writer = ExampleParquetWriter.builder(HadoopOutputFile.fromPath(hPath, conf)) + .withConf(conf) + .build()) { + SimpleGroupFactory f = new SimpleGroupFactory(schema); + writer.write(f.newGroup() + .append("name", "Alice") + .append("age", 31)); + writer.write(f.newGroup() + .append("name", "Bob") + .append("age", 25)); + } + + FilterPredicate pred = FilterApi.gt(FilterApi.intColumn("age"), 30); + List selected = new ArrayList<>(); + + try (ParquetReader reader = ParquetReader.builder(new GroupReadSupport(), hPath) + .withConf(conf) + .withFilter(FilterCompat.get(pred)) + .build()) { + Group g; + while ((g = reader.read()) != null) { + selected.add(g.getBinary("name", 0) + .toStringUsingUTF8()); + } + } + + assertEquals(List.of("Alice"), selected); + } + } + + @Nested + class WriterOptionsUnitTest { + + @Test + void givenWriterOptions_whenBuildingWriter_thenItUsesZstdAndDictionary(@TempDir java.nio.file.Path tmp) throws Exception { + Path hPath = new Path(tmp.resolve("opts.parquet").toUri()); + MessageType schema = MessageTypeParser.parseMessageType("message m { required binary name (UTF8); required int32 age; }"); + Configuration conf = new Configuration(); + OutputFile out = HadoopOutputFile.fromPath(hPath, conf); + + SimpleGroupFactory factory = new SimpleGroupFactory(schema); + + try (ParquetWriter writer = ExampleParquetWriter.builder(out) + .withType(schema) + .withConf(conf) + .withCompressionCodec(CompressionCodecName.ZSTD) + .withDictionaryEncoding(true) + .withPageSize(ParquetProperties.DEFAULT_PAGE_SIZE) + .build()) { + String[] names = { "alice", "bob", "carol", "dave", "erin" }; + int[] ages = { 30, 31, 32, 33, 34 }; + for (int i = 0; i < 5000; i++) { + String n = names[i % names.length]; + int a = ages[i % ages.length]; + writer.write(factory.newGroup() + .append("name", n) + .append("age", a)); + } + } + + ParquetMetadata meta; + try (ParquetFileReader reader = ParquetFileReader.open(HadoopInputFile.fromPath(hPath, conf))) { + meta = reader.getFooter(); + } + + assertFalse(meta.getBlocks().isEmpty()); + + boolean nameColumnUsedDictionary = false; + + for (BlockMetaData block : meta.getBlocks()) { + assertFalse(block.getColumns().isEmpty()); + for (ColumnChunkMetaData col : block.getColumns()) { + assertEquals(CompressionCodecName.ZSTD, col.getCodec()); + if ("name".equals(col.getPath().toDotString())) { + EncodingStats stats = col.getEncodingStats(); + boolean dictByStats = stats != null && stats.hasDictionaryEncodedPages(); + Set enc = col.getEncodings(); + boolean dictByEncSet = enc.contains(Encoding.RLE_DICTIONARY) || enc.contains(Encoding.DELTA_BYTE_ARRAY); + boolean dictPagePresent = col.hasDictionaryPage(); + if (dictByStats || dictByEncSet || dictPagePresent) { + nameColumnUsedDictionary = true; + } + } + } + } + + assertTrue(nameColumnUsedDictionary); + } + + @Test + void givenCompressionAndDictionary_whenComparingSizes_thenOptimizedIsSmaller(@TempDir java.nio.file.Path tmp) throws Exception { + MessageType schema = MessageTypeParser.parseMessageType(""" + message m { + required binary name (UTF8); + required int32 age; + } + """); + + Configuration conf = new Configuration(); + SimpleGroupFactory factory = new SimpleGroupFactory(schema); + + java.nio.file.Path baselineNio = tmp.resolve("people-baseline.parquet"); + java.nio.file.Path optimizedNio = tmp.resolve("people-optimized.parquet"); + + Path baseline = new Path(baselineNio.toUri()); + Path optimized = new Path(optimizedNio.toUri()); + + String[] names = { "alice", "bob", "carol", "dave", "erin" }; + int[] ages = { 30, 31, 32, 33, 34 }; + int rows = 5000; + + try (ParquetWriter w = ExampleParquetWriter + .builder(HadoopOutputFile.fromPath(baseline, conf)) + .withType(schema) + .withConf(conf) + .withCompressionCodec(CompressionCodecName.UNCOMPRESSED) + .withDictionaryEncoding(false) + .build()) { + for (int i = 0; i < rows; i++) { + w.write(factory.newGroup() + .append("name", names[i % names.length]) + .append("age", ages[i % ages.length])); + } + } + + try (ParquetWriter w = ExampleParquetWriter + .builder(HadoopOutputFile.fromPath(optimized, conf)) + .withType(schema) + .withConf(conf) + .withCompressionCodec(CompressionCodecName.ZSTD) + .withDictionaryEncoding(true) + .build()) { + for (int i = 0; i < rows; i++) { + w.write(factory.newGroup() + .append("name", names[i % names.length]) + .append("age", ages[i % ages.length])); + } + } + + long baselineBytes = Files.size(baselineNio); + long optimizedBytes = Files.size(optimizedNio); + + long saved = baselineBytes - optimizedBytes; + double pct = (baselineBytes == 0) ? 0.0 : (saved * 100.0) / baselineBytes; + + Logger log = Logger.getLogger("parquet.tutorial"); + log.info(String.format("Baseline: %,d bytes; Optimized: %,d bytes; Saved: %,d bytes (%.1f%%)", + baselineBytes, optimizedBytes, saved, pct)); + + assertTrue(optimizedBytes < baselineBytes, "Optimized file should be smaller than baseline"); + } + } +} diff --git a/apache-libraries-3/src/test/java/com/baeldung/apache/parquet/givenSchema_whenWritingAndReadingWithExampleApi_thenRoundtripWorks.puml b/apache-libraries-3/src/test/java/com/baeldung/apache/parquet/givenSchema_whenWritingAndReadingWithExampleApi_thenRoundtripWorks.puml new file mode 100644 index 000000000000..aa71e5df74a4 --- /dev/null +++ b/apache-libraries-3/src/test/java/com/baeldung/apache/parquet/givenSchema_whenWritingAndReadingWithExampleApi_thenRoundtripWorks.puml @@ -0,0 +1,131 @@ +@startuml +title Parquet Example API Round-trip + +skinparam monochrome true +skinparam activity { + BackgroundColor White + BorderColor Black +} +skinparam note { + BorderColor Black + BackgroundColor #F8F8F8 +} + +start + +partition "Setup" { + :Parse schema string -> MessageType + (MessageTypeParser.parseMessageType); + note right + Parquet schema in its own IDL: + - "message person" is the root record. + - required vs optional = nullability. + - binary (UTF8) = physical binary with + a logical 'string' annotation. + - int32 = 32-bit integer. + The schema governs both writer and reader. + end note + + :Create SimpleGroupFactory(schema); + note right + Group = generic, untyped row container. + SimpleGroupFactory enforces the MessageType + when appending values. Repeated fields would + use positional indexes; here all fields are + non-repeated (index 0). + end note + + :Create Hadoop Configuration; + :Resolve temp file Path (Hadoop Path); + note right + Hadoop Path + Configuration work on the local + filesystem in tests, but also abstract HDFS/S3/etc. + end note +} + +partition "Write path" { + :Build ExampleParquetWriter + - HadoopOutputFile.fromPath(path, conf) + - withType(schema) + - withConf(conf); + note right + The writer binds to the schema and target file. + Options like compression (e.g., ZSTD/Snappy), + dictionary encoding, page/row-group sizes can be + set here in real projects. + try-with-resources guarantees the footer and + metadata are flushed. + end note + + :Create Group #1 + (name="Alice", age=34, city="Rome"); + note right + Values must match the schema types and logical + annotations (UTF-8 for strings). Nullability: 'city' + is optional, so it may be present or absent. + end note + + :Create Group #2 + (name="Bob", age=29); + note right + Omitting 'city' is legal because it's optional. + If a field were required and missing, the writer + would fail validation. + end note + + :writer.write(group1); + :writer.write(group2); + + :Close writer (try-with-resources); +} + +partition "Read path" { + :Build ParquetReader + - GroupReadSupport + - path + conf; + note right + GroupReadSupport reconstructs Group records. + We can configure projection (read subset of + columns) and filters to reduce I/O. Here we + read all columns for simplicity. + end note + + :Initialize empty List names; + :Declare Group g; + :g = reader.read(); + + while (g != null?) + :Extract name = + g.getBinary("name", 0) + .toStringUsingUTF8(); + note right + Access by field name and index (0 for + non-repeated fields). 'getBinary' returns + the physical type; 'toStringUsingUTF8' + respects the logical string annotation. + end note + :names.add(name); + :g = reader.read(); + endwhile + note right + reader.read() returns null at EOF (no exception). + The loop materializes rows one by one; with + projection enabled, non-requested columns + are never decoded. + end note + + :Close reader; +} + +partition "Assertion" { + :Assert names == ["Alice","Bob"]; + note right + Simple round-trip check: what we wrote is exactly + what we read. Sequential readers return rows in the + original write order. + end note +} + +stop +@enduml + diff --git a/apache-libraries/README.md b/apache-libraries/README.md deleted file mode 100644 index ef6415f980e4..000000000000 --- a/apache-libraries/README.md +++ /dev/null @@ -1,13 +0,0 @@ -## Apache Libraries - -This module contains articles about various Apache libraries and utilities - -### Relevant Articles: -- [Guide to Apache Avro](https://www.baeldung.com/java-apache-avro) -- [Intro to Apache OpenNLP](https://www.baeldung.com/apache-open-nlp) -- [Getting Started with Java and Zookeeper](https://www.baeldung.com/java-zookeeper) -- [Guide To Solr in Java With Apache SolrJ](https://www.baeldung.com/apache-solrj) -- [Understanding XSLT Processing in Java](https://www.baeldung.com/java-extensible-stylesheet-language-transformations) -- [Create Avro Schema With List of Objects](https://www.baeldung.com/avro-schema-list-objects) -- [A Guide to Apache Mesos](https://www.baeldung.com/apache-mesos) -- More articles: [[next -->]](../apache-libraries-2) diff --git a/apache-libraries/pom.xml b/apache-libraries/pom.xml index 30d9fe5e8343..de8d32b843e3 100644 --- a/apache-libraries/pom.xml +++ b/apache-libraries/pom.xml @@ -78,7 +78,6 @@ mesos ${mesos.library.version} - net.javacrumbs.json-unit json-unit-assertj @@ -114,6 +113,7 @@ 15 15 + 15 @@ -121,7 +121,8 @@ 3.0.1 - 3.1.01.8.4 + 3.1.0 + 1.8.4 6.4.0 1.12.0 1.11.0 diff --git a/apache-maven-4/pom.xml b/apache-maven-4/pom.xml new file mode 100644 index 000000000000..14f151a1fbd2 --- /dev/null +++ b/apache-maven-4/pom.xml @@ -0,0 +1,82 @@ + + + + com.baeldung + apache-maven-4 + 1.0-SNAPSHOT + pom + + + + + org.junit + junit-bom + ${junit-bom.version} + pom + import + + + org.apache.logging.log4j + log4j-core + ${log4j-core.version} + + + org.projectlombok + lombok + ${lombok.version} + provided + + + + + + + org.projectlombok + lombok + ${lombok.version} + classpath-processor + + + + + + + + + maven-compiler-plugin + ${maven-compiler-plugin.version} + + + maven-install-plugin + ${maven-install-plugin.version} + + + + + + + + conditional-profile + + + 5]]> + + + + + + + 17 + 17 + 5.13.4 + 2.24.3 + 1.18.42 + + 4.0.0-beta-3 + 4.0.0-beta-2 + + + diff --git a/apache-maven-4/project-a/pom.xml b/apache-maven-4/project-a/pom.xml new file mode 100644 index 000000000000..187589b232b4 --- /dev/null +++ b/apache-maven-4/project-a/pom.xml @@ -0,0 +1,24 @@ + + + + com.baeldung.apache-maven-4 + project-a + + + + + + org.junit.jupiter + junit-jupiter-engine + test + + + org.apache.logging.log4j + log4j-core + + + + diff --git a/apache-maven-4/project-a/src/main/java/com/example/App.java b/apache-maven-4/project-a/src/main/java/com/example/App.java new file mode 100644 index 000000000000..8fea7112ec37 --- /dev/null +++ b/apache-maven-4/project-a/src/main/java/com/example/App.java @@ -0,0 +1,13 @@ +package com.example; + +/** + * Hello world! + * + */ +public class App { + + public static void main(String[] args) { + System.out.println("Hello World!"); + } + +} diff --git a/apache-maven-4/project-b/pom.xml b/apache-maven-4/project-b/pom.xml new file mode 100644 index 000000000000..7c14bcaf77d0 --- /dev/null +++ b/apache-maven-4/project-b/pom.xml @@ -0,0 +1,19 @@ + + + + com.baeldung.apache-maven-4 + project-b + + + + + + org.projectlombok + lombok + + + + diff --git a/apache-maven-4/project-b/src/main/java/com/example/App.java b/apache-maven-4/project-b/src/main/java/com/example/App.java new file mode 100644 index 000000000000..1f0151bd69e6 --- /dev/null +++ b/apache-maven-4/project-b/src/main/java/com/example/App.java @@ -0,0 +1,12 @@ +package com.example; + +/** + * Hello world! + * + */ +public class App { + public static void main(String[] args) { + System.out.println("Hello World!"); + new Person().setName("Jack"); + } +} diff --git a/apache-maven-4/project-b/src/main/java/com/example/Person.java b/apache-maven-4/project-b/src/main/java/com/example/Person.java new file mode 100644 index 000000000000..44fb73736a8c --- /dev/null +++ b/apache-maven-4/project-b/src/main/java/com/example/Person.java @@ -0,0 +1,10 @@ +package com.example; + +import lombok.Data; + +@Data +public class Person { + + private String name; + +} diff --git a/apache-olingo/README.md b/apache-olingo/README.md deleted file mode 100644 index 2f4e86d5a2ad..000000000000 --- a/apache-olingo/README.md +++ /dev/null @@ -1,8 +0,0 @@ -## Apache Olingo - -This module contains articles about Apache Olingo - -### Relevant articles: - -- [OData Protocol Guide](https://www.baeldung.com/odata) -- [Intro to OData with Olingo](https://www.baeldung.com/olingo) diff --git a/apache-poi-2/README.md b/apache-poi-2/README.md deleted file mode 100644 index 9638363f9165..000000000000 --- a/apache-poi-2/README.md +++ /dev/null @@ -1,16 +0,0 @@ -## Apache POI - -This module contains articles about Apache POI. - -### Relevant Articles: - -- [Adding a Column to an Excel Sheet Using Apache POI](https://www.baeldung.com/java-excel-add-column) -- [Add an Image to a Cell in an Excel File With Java](https://www.baeldung.com/java-add-image-excel) -- [Numeric Format Using POI](https://www.baeldung.com/apache-poi-numeric-format) -- [Creating a MS PowerPoint Presentation in Java](https://www.baeldung.com/apache-poi-slideshow) -- [Finding the Last Row in an Excel Spreadsheet From Java](https://www.baeldung.com/java-excel-find-last-row) -- [Setting Formulas in Excel with Apache POI](https://www.baeldung.com/java-apache-poi-set-formulas) -- [Set the Date Format Using Apache POI](https://www.baeldung.com/java-apache-poi-date-format) -- [Replacing Variables in a Document Template with Java](https://www.baeldung.com/java-replace-pattern-word-document-doc-docx) -- [Lock Header Rows With Apache POI](https://www.baeldung.com/java-apache-poi-lock-header-rows) -- More articles: [[<-- prev]](../apache-poi)[[next -->]](../apache-poi-3) diff --git a/apache-poi-3/README.md b/apache-poi-3/README.md deleted file mode 100644 index 4de8dc61a6ae..000000000000 --- a/apache-poi-3/README.md +++ /dev/null @@ -1,14 +0,0 @@ -## Apache POI - -This module contains articles about Apache POI. - -### Relevant Articles: - -- [Apply Bold Text Style for an Entire Row Using Apache POI](https://www.baeldung.com/appache-poi-apply-bold-text-style-entire-row) -- [Using Apache POI to Extract Column Names From Excel](https://www.baeldung.com/apache-poi-extract-column-names-excel) -- [Generate MS Word Documents Using poi-tl Template](https://www.baeldung.com/poi-tl-ms-word) -- [Add Borders to Excel Cells With Apache POI](https://www.baeldung.com/apache-poi-add-borders) -- [Insert a Row in Excel Using Apache POI](https://www.baeldung.com/apache-poi-insert-excel-row) -- [Writing JDBC ResultSet to an Excel File Using Apache POI](https://www.baeldung.com/jdbc-resultset-excel) - -- More articles: [[<-- prev]](../apache-poi-2) diff --git a/apache-poi-3/pom.xml b/apache-poi-3/pom.xml index 25b3e7fa6073..675cff2fb10a 100644 --- a/apache-poi-3/pom.xml +++ b/apache-poi-3/pom.xml @@ -14,6 +14,11 @@ + + org.springframework.boot + spring-boot-starter-web + ${spring.web.version} + org.apache.poi poi-ooxml @@ -89,6 +94,11 @@ jmh-generator-annprocess ${jmh.version} + + org.apache.logging.log4j + log4j-core + ${log4j.version} + @@ -102,6 +112,8 @@ 1.5.6 1.5.6 1.37 + 2.23.1 + 3.5.7 - \ No newline at end of file + diff --git a/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/DemoApplication.java b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/DemoApplication.java new file mode 100644 index 000000000000..63195983fe0f --- /dev/null +++ b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/DemoApplication.java @@ -0,0 +1,16 @@ +package com.baeldung.hssfworkbook; + +import java.io.IOException; + +import org.apache.poi.hssf.usermodel.HSSFWorkbook; +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +@SpringBootApplication +public class DemoApplication { + + public static void main(String[] args) throws IOException { + SpringApplication.run(DemoApplication.class, args); + } + +} diff --git a/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelController.java b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelController.java new file mode 100644 index 000000000000..98e91f0412dd --- /dev/null +++ b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelController.java @@ -0,0 +1,65 @@ +package com.baeldung.hssfworkbook; + +import org.apache.poi.hssf.usermodel.HSSFWorkbook; +import org.apache.poi.ss.usermodel.Sheet; +import org.springframework.http.HttpHeaders; +import org.springframework.http.MediaType; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.RestController; +import org.springframework.web.multipart.MultipartFile; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; + +@RestController +public class ExcelController { + + @GetMapping("/download-excel") + public ResponseEntity downloadExcel() { + try { + HSSFWorkbook workbook = ExcelCreator.createSampleWorkbook(); + + try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) { + workbook.write(baos); + byte[] bytes = baos.toByteArray(); + + return ResponseEntity.ok() + .header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=employee_data.xls") + .contentType(MediaType.parseMediaType("application/vnd.ms-excel")) // More specific MIME type + .body(bytes); + } + } catch (IOException e) { + System.err.println("Error generating or writing Excel workbook: " + e.getMessage()); + return ResponseEntity.internalServerError() + .build(); + } + } + + @PostMapping("/upload-excel") + public ResponseEntity uploadExcel(@RequestParam("file") MultipartFile file) { + if (file.isEmpty()) { + return ResponseEntity.badRequest() + .body("File is empty. Please upload a file."); + } + + try { + try (HSSFWorkbook workbook = new HSSFWorkbook(file.getInputStream())) { + Sheet sheet = workbook.getSheetAt(0); + + return ResponseEntity.ok("Sheet '" + sheet.getSheetName() + "' uploaded successfully!"); + } + } catch (IOException e) { + System.err.println("Error processing uploaded Excel file: " + e.getMessage()); + return ResponseEntity.internalServerError() + .body("Failed to process the Excel file."); + } catch (Exception e) { + System.err.println("An unexpected error occurred during file upload: " + e.getMessage()); + return ResponseEntity.internalServerError() + .body("An unexpected error occurred."); + } + } + +} diff --git a/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelConverter.java b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelConverter.java new file mode 100644 index 000000000000..39db5bd2130d --- /dev/null +++ b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelConverter.java @@ -0,0 +1,23 @@ +package com.baeldung.hssfworkbook; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; + +import org.apache.poi.hssf.usermodel.HSSFWorkbook; + +public class ExcelConverter { + + public static byte[] convertWorkbookToBytes(HSSFWorkbook workbook) throws IOException { + try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) { + workbook.write(baos); + return baos.toByteArray(); + } + } + + public static HSSFWorkbook convertBytesToWorkbook(byte[] excelBytes) throws IOException { + try (ByteArrayInputStream bais = new ByteArrayInputStream(excelBytes)) { + return new HSSFWorkbook(bais); + } + } +} diff --git a/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelCreator.java b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelCreator.java new file mode 100644 index 000000000000..07c5dfea62cf --- /dev/null +++ b/apache-poi-3/src/main/java/com/baeldung/hssfworkbook/ExcelCreator.java @@ -0,0 +1,56 @@ +package com.baeldung.hssfworkbook; + +import org.apache.poi.hssf.usermodel.HSSFCellStyle; +import org.apache.poi.hssf.usermodel.HSSFFont; +import org.apache.poi.hssf.usermodel.HSSFWorkbook; +import org.apache.poi.ss.usermodel.Cell; +import org.apache.poi.ss.usermodel.Row; +import org.apache.poi.ss.usermodel.Sheet; + +public class ExcelCreator { + + public static HSSFWorkbook createSampleWorkbook() { + HSSFWorkbook workbook = new HSSFWorkbook(); + + final String SHEET_NAME = "Employees"; + final String[] COLUMN_HEADERS = { "ID", "Name", "Department" }; + Object[][] data = { { 101, "John Doe", "Finance" }, { 102, "Jane Smith", "HR" }, { 103, "Michael Clark", "IT" } }; + + Sheet sheet = workbook.createSheet(SHEET_NAME); + + HSSFFont font = workbook.createFont(); + font.setBold(true); + HSSFCellStyle headerStyle = workbook.createCellStyle(); + headerStyle.setFont(font); + + Row header = sheet.createRow(0); + for (int i = 0; i < COLUMN_HEADERS.length; i++) { + Cell cell = header.createCell(i); + cell.setCellValue(COLUMN_HEADERS[i]); + cell.setCellStyle(headerStyle); + } + + int rowNum = 1; + for (Object[] rowData : data) { + Row row = sheet.createRow(rowNum++); + for (int i = 0; i < rowData.length; i++) { + Cell cell = row.createCell(i); + Object value = rowData[i]; + + if (value instanceof Integer) { + cell.setCellValue(((Integer) value).doubleValue()); + } else if (value instanceof Double) { + cell.setCellValue((Double) value); + } else if (value != null) { + cell.setCellValue(value.toString()); + } + } + } + + for (int i = 0; i < COLUMN_HEADERS.length; i++) { + sheet.autoSizeColumn(i); + } + + return workbook; + } +} diff --git a/apache-poi-3/src/test/java/com/baeldung/hssfworkbook/ExcelConverterUnitTest.java b/apache-poi-3/src/test/java/com/baeldung/hssfworkbook/ExcelConverterUnitTest.java new file mode 100644 index 000000000000..31240663f250 --- /dev/null +++ b/apache-poi-3/src/test/java/com/baeldung/hssfworkbook/ExcelConverterUnitTest.java @@ -0,0 +1,86 @@ +package com.baeldung.hssfworkbook; + +import org.apache.poi.hssf.usermodel.HSSFWorkbook; +import org.apache.poi.ss.usermodel.Sheet; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.UncheckedIOException; + +import com.baeldung.hssfworkbook.ExcelConverter; + +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.assertEquals; + +class ExcelConverterTest { + + private HSSFWorkbook createMinimalWorkbook() { + HSSFWorkbook workbook = new HSSFWorkbook(); + Sheet sheet = workbook.createSheet("TestSheet"); + sheet.createRow(0) + .createCell(0) + .setCellValue("Test Data"); + return workbook; + } + + private byte[] convertWorkbookToBytesSafely(HSSFWorkbook workbook) { + try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) { + workbook.write(baos); + return baos.toByteArray(); + } catch (IOException e) { + throw new UncheckedIOException("Failed to create test bytes.", e); + } + } + + @Test + @DisplayName("GivenAValidHSSFWorkbook_whenConvertingToBytes_thenByteArrayIsNonEmptyAndValid") + void givenAValidHSSFWorkbook_whenConvertingToBytes_thenByteArrayIsNonEmptyAndValid() throws IOException { + HSSFWorkbook workbook = createMinimalWorkbook(); + + byte[] resultBytes = ExcelConverter.convertWorkbookToBytes(workbook); + + assertNotNull(resultBytes, "The byte array should not be null."); + assertTrue(resultBytes.length > 0, "The byte array should not be empty."); + + HSSFWorkbook convertedWorkbook = ExcelConverter.convertBytesToWorkbook(resultBytes); + assertEquals("TestSheet", convertedWorkbook.getSheetName(0)); + } + + @Test + @DisplayName("GivenAValidExcelByteArray_whenConvertingToWorkbook_thenHSSFWorkbookIsReturned") + void givenAValidExcelByteArray_whenConvertingToWorkbook_thenHSSFWorkbookIsReturned() throws IOException { + HSSFWorkbook originalWorkbook = createMinimalWorkbook(); + byte[] validExcelBytes = convertWorkbookToBytesSafely(originalWorkbook); + + HSSFWorkbook resultWorkbook = ExcelConverter.convertBytesToWorkbook(validExcelBytes); + + assertNotNull(resultWorkbook, "The resulting workbook should not be null."); + assertEquals(1, resultWorkbook.getNumberOfSheets(), "The resulting workbook should have 1 sheet."); + assertEquals("TestSheet", resultWorkbook.getSheetName(0), "Sheet name should match the original."); + assertEquals("Test Data", resultWorkbook.getSheetAt(0) + .getRow(0) + .getCell(0) + .getStringCellValue()); + } + + @Test + @DisplayName("GivenAnEmptyByteArray_whenConvertingToWorkbook_thenIOExceptionIsThrown") + void givenAnEmptyByteArray_whenConvertingToWorkbook_thenIOExceptionIsThrown() { + byte[] emptyBytes = new byte[0]; + + assertThrows(IOException.class, () -> ExcelConverter.convertBytesToWorkbook(emptyBytes), "Expected IOException for empty byte array."); + } + + @Test + @DisplayName("GivenAnInvalidByteArray_whenConvertingToWorkbook_thenIOExceptionIsThrown") + void givenAnInvalidByteArray_whenConvertingToWorkbook_thenIOExceptionIsThrown() { + byte[] invalidBytes = { 0x01, 0x02, 0x03, 0x04, 0x05 }; + + assertThrows(IOException.class, () -> ExcelConverter.convertBytesToWorkbook(invalidBytes), "Expected IOException for invalid byte array format."); + } +} \ No newline at end of file diff --git a/apache-poi-3/src/test/java/com/baeldung/hssfworkbook/ExcelCreatorUnitTest.java b/apache-poi-3/src/test/java/com/baeldung/hssfworkbook/ExcelCreatorUnitTest.java new file mode 100644 index 000000000000..ceadb2d3b296 --- /dev/null +++ b/apache-poi-3/src/test/java/com/baeldung/hssfworkbook/ExcelCreatorUnitTest.java @@ -0,0 +1,94 @@ +package com.baeldung.hssfworkbook; + +import org.apache.poi.hssf.usermodel.HSSFWorkbook; +import org.apache.poi.ss.usermodel.Cell; +import org.apache.poi.ss.usermodel.CellStyle; +import org.apache.poi.ss.usermodel.CellType; +import org.apache.poi.ss.usermodel.Font; +import org.apache.poi.ss.usermodel.Row; +import org.apache.poi.ss.usermodel.Sheet; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +import com.baeldung.hssfworkbook.ExcelCreator; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class ExcelCreatorTest { + + private static final String SHEET_NAME = "Employees"; + private static final String[] COLUMN_HEADERS = { "ID", "Name", "Department" }; + private static final Object[][] DATA = { { 101, "John Doe", "Finance" }, { 102, "Jane Smith", "HR" }, { 103, "Michael Clark", "IT" } }; + + @Test + @DisplayName("Workbook should be created and contain one sheet named 'Employees'") + void givenExcelCreator_whenCreateSampleWorkbookCalled_thenWorkbookContainsOneSheetNamedEmployees() { + HSSFWorkbook workbook = ExcelCreator.createSampleWorkbook(); + + assertNotNull(workbook, "The workbook should not be null."); + assertEquals(1, workbook.getNumberOfSheets(), "The workbook should contain exactly one sheet."); + + Sheet sheet = workbook.getSheet(SHEET_NAME); + assertNotNull(sheet, "The sheet named '" + SHEET_NAME + "' must exist."); + } + + @Test + @DisplayName("Header row should have correct content and bold style") + void givenExcelCreator_whenCreateSampleWorkbookCalled_thenHeaderRowContainsCorrectTextAndBoldStyle() { + HSSFWorkbook workbook = ExcelCreator.createSampleWorkbook(); + Sheet sheet = workbook.getSheet(SHEET_NAME); + + Row headerRow = sheet.getRow(0); + assertNotNull(headerRow, "Header row (row 0) should exist."); + + Font expectedFont = workbook.getFontAt(headerRow.getCell(0) + .getCellStyle() + .getFontIndex()); + assertTrue(expectedFont.getBold(), "Header font should be bold."); + + for (int i = 0; i < COLUMN_HEADERS.length; i++) { + Cell cell = headerRow.getCell(i); + assertNotNull(cell, "Header cell at index " + i + " should exist."); + assertEquals(COLUMN_HEADERS[i], cell.getStringCellValue(), "Header text mismatch at column " + i + "."); + + CellStyle cellStyle = cell.getCellStyle(); + assertNotNull(cellStyle, "Header cell should have a style applied."); + + Font actualFont = workbook.getFontAt(cellStyle.getFontIndex()); + assertTrue(actualFont.getBold(), "Header cell at index " + i + " style must be bold."); + } + } + + @Test + @DisplayName("Data rows should have correct content and data type handling") + void givenExcelCreator_whenCreateSampleWorkbookCalled_thenDataRowsContainCorrectContentAndTypes() { + HSSFWorkbook workbook = ExcelCreator.createSampleWorkbook(); + Sheet sheet = workbook.getSheet(SHEET_NAME); + + assertEquals(DATA.length + 1, sheet.getLastRowNum() + 1, "The sheet should contain header row + data rows."); + + for (int i = 0; i < DATA.length; i++) { + Row dataRow = sheet.getRow(i + 1); + Object[] expectedRowData = DATA[i]; + + assertNotNull(dataRow, "Data row at index " + (i + 1) + " should exist."); + + for (int j = 0; j < expectedRowData.length; j++) { + Cell cell = dataRow.getCell(j); + Object expectedValue = expectedRowData[j]; + + assertNotNull(cell, "Cell [" + (i + 1) + ", " + j + "] should exist."); + + if (expectedValue instanceof Integer) { + assertEquals(CellType.NUMERIC, cell.getCellType(), "Cell " + j + " type should be NUMERIC for Integer."); + assertEquals(((Integer) expectedValue).doubleValue(), cell.getNumericCellValue(), 0.001, "Numeric value mismatch."); + } else { + assertEquals(CellType.STRING, cell.getCellType(), "Cell " + j + " type should be STRING."); + assertEquals(expectedValue.toString(), cell.getStringCellValue(), "String value mismatch."); + } + } + } + } +} \ No newline at end of file diff --git a/apache-poi-4/.gitignore b/apache-poi-4/.gitignore new file mode 100644 index 000000000000..60b67aab8f8a --- /dev/null +++ b/apache-poi-4/.gitignore @@ -0,0 +1 @@ +CellStyleTest_output.xlsx \ No newline at end of file diff --git a/apache-poi-4/CellStyleTest_output.xlsx b/apache-poi-4/CellStyleTest_output.xlsx deleted file mode 100644 index 31b1dbec2e03..000000000000 Binary files a/apache-poi-4/CellStyleTest_output.xlsx and /dev/null differ diff --git a/apache-poi-4/README.md b/apache-poi-4/README.md deleted file mode 100644 index 85012a60f171..000000000000 --- a/apache-poi-4/README.md +++ /dev/null @@ -1,12 +0,0 @@ -## Apache POI - -This module contains articles about Apache POI. - -### Relevant Articles: - -- [Change Cell Font Style with Apache POI](https://www.baeldung.com/apache-poi-change-cell-font) -- [Merge Cells in Excel Using Apache POI](https://www.baeldung.com/java-apache-poi-merge-cells) -- [Read Excel Cell Value Rather Than Formula With Apache POI](https://www.baeldung.com/apache-poi-read-cell-value-formula) -- [Get String Value of Excel Cell with Apache POI](https://www.baeldung.com/java-apache-poi-cell-string-value) - -- More articles: [[<-- prev]](../apache-poi-3) diff --git a/apache-poi-4/pom.xml b/apache-poi-4/pom.xml index cb9058ba3386..7fa8227cf382 100644 --- a/apache-poi-4/pom.xml +++ b/apache-poi-4/pom.xml @@ -44,6 +44,11 @@ commons-collections4 ${commons-collections4.version} + + fr.opensagres.xdocreport + fr.opensagres.poi.xwpf.converter.xhtml + ${fr.opensagres.poi.xwpf.converter.xhtml.version} + ch.qos.logback logback-classic @@ -53,7 +58,12 @@ ch.qos.logback logback-core ${logback-core.version} - + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + @@ -61,7 +71,9 @@ 4.1.2 5.2.0 1.5.6 - 1.5.6 + 1.5.6 + 2.23.1 + 2.1.0 \ No newline at end of file diff --git a/apache-poi-4/src/main/java/com/baeldung/poi/html/DocToHtml.java b/apache-poi-4/src/main/java/com/baeldung/poi/html/DocToHtml.java new file mode 100644 index 000000000000..87f7afa08f23 --- /dev/null +++ b/apache-poi-4/src/main/java/com/baeldung/poi/html/DocToHtml.java @@ -0,0 +1,54 @@ +package com.baeldung.poi.html; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; + +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.transform.OutputKeys; +import javax.xml.transform.Transformer; +import javax.xml.transform.TransformerFactory; +import javax.xml.transform.dom.DOMSource; +import javax.xml.transform.stream.StreamResult; + +import org.apache.poi.hwpf.HWPFDocumentCore; +import org.apache.poi.hwpf.converter.WordToHtmlConverter; +import org.apache.poi.hwpf.converter.WordToHtmlUtils; +import org.w3c.dom.Document; + +public class DocToHtml { + +public void convertDocToHtml(String docPath) throws Exception { + Path input = Paths.get(docPath); + String htmlFileName = input.getFileName().toString().replaceFirst("\\.[^.]+$", "") + ".html"; + Path output = input.resolveSibling(htmlFileName); + Path imagesDir = input.resolveSibling("images"); + Files.createDirectories(imagesDir); + try (InputStream in = Files.newInputStream(Paths.get(docPath)); OutputStream out = Files.newOutputStream(output)) { + HWPFDocumentCore document = WordToHtmlUtils.loadDoc(in); + Document htmlDocument = DocumentBuilderFactory.newInstance() + .newDocumentBuilder() + .newDocument(); + WordToHtmlConverter converter = new WordToHtmlConverter(htmlDocument); + converter.setPicturesManager((content, pictureType, suggestedName, widthInches, heightInches) -> { + Path imageFile = imagesDir.resolve(suggestedName); + try { + Files.write(imageFile, content); + } catch (IOException e) { + throw new RuntimeException(e); + } + return "images/" + suggestedName; + }); + converter.processDocument(document); + Transformer transformer = TransformerFactory.newInstance() + .newTransformer(); + transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8"); + transformer.setOutputProperty(OutputKeys.METHOD, "html"); + transformer.transform(new DOMSource(converter.getDocument()), new StreamResult(out)); + } +} + +} diff --git a/apache-poi-4/src/main/java/com/baeldung/poi/html/DocxToHtml.java b/apache-poi-4/src/main/java/com/baeldung/poi/html/DocxToHtml.java new file mode 100644 index 000000000000..ceb9d8fbac79 --- /dev/null +++ b/apache-poi-4/src/main/java/com/baeldung/poi/html/DocxToHtml.java @@ -0,0 +1,51 @@ +package com.baeldung.poi.html; + +import fr.opensagres.poi.xwpf.converter.core.ImageManager; +import fr.opensagres.poi.xwpf.converter.xhtml.XHTMLConverter; +import fr.opensagres.poi.xwpf.converter.xhtml.XHTMLOptions; +import org.apache.poi.xwpf.usermodel.XWPFDocument; + +import java.io.*; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.nio.file.StandardOpenOption; + +public class DocxToHtml { + + public XWPFDocument loadDocxFromPath(String path) { + try { + Path file = Paths.get(path); + if (!Files.exists(file)) { + throw new FileNotFoundException("File not found: " + path); + } + XWPFDocument document = new XWPFDocument(Files.newInputStream(file)); + boolean hasParagraphs = !document.getParagraphs().isEmpty(); + boolean hasTables = !document.getTables().isEmpty(); + if (!hasParagraphs && !hasTables) { + document.close(); + throw new IllegalArgumentException("Document is empty: " + path); + } + return document; + } catch (IOException ex) { + throw new UncheckedIOException("Cannot load document: " + path, ex); + } + } + + private XHTMLOptions configureHtmlOptions(Path outputDir) { + XHTMLOptions options = XHTMLOptions.create(); + options.setImageManager(new ImageManager(outputDir.toFile(), "images")); + return options; + } + + public void convertDocxToHtml(String docxPath) throws IOException { + Path input = Paths.get(docxPath); + String htmlFileName = input.getFileName().toString().replaceFirst("\\.[^.]+$", "") + ".html"; + Path output = input.resolveSibling(htmlFileName); + try (XWPFDocument document = loadDocxFromPath(docxPath); OutputStream out = Files.newOutputStream(output)) { + XHTMLConverter.getInstance().convert(document, out, configureHtmlOptions(output.getParent())); + } + } + +} diff --git a/apache-poi-4/src/main/resources/sample.doc b/apache-poi-4/src/main/resources/sample.doc new file mode 100644 index 000000000000..9423c5a41418 Binary files /dev/null and b/apache-poi-4/src/main/resources/sample.doc differ diff --git a/apache-poi-4/src/main/resources/sample.docx b/apache-poi-4/src/main/resources/sample.docx new file mode 100644 index 000000000000..759f95a87db9 Binary files /dev/null and b/apache-poi-4/src/main/resources/sample.docx differ diff --git a/apache-poi-4/src/test/java/com/baeldung/poi/html/DocToHtmlUnitTest.java b/apache-poi-4/src/test/java/com/baeldung/poi/html/DocToHtmlUnitTest.java new file mode 100644 index 000000000000..e4ab168a1496 --- /dev/null +++ b/apache-poi-4/src/test/java/com/baeldung/poi/html/DocToHtmlUnitTest.java @@ -0,0 +1,30 @@ +package com.baeldung.poi.html; + +import static org.junit.jupiter.api.Assertions.*; + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.stream.Collectors; + +import org.junit.jupiter.api.Test; + +class DocToHtmlUnitTest { + @Test + void givenSimpleDoc_whenConverting_thenHtmlFileIsCreated() throws Exception { + DocToHtml converter = new DocToHtml(); + Path doc = Paths.get(this.getClass() + .getResource("/sample.doc") + .getPath()); + + converter.convertDocToHtml(doc.toString()); + + Path html = doc.resolveSibling("sample.html"); + assertTrue(Files.exists(html)); + + String content = Files.lines(html, StandardCharsets.UTF_8) + .collect(Collectors.joining("\n")); + assertTrue(content.contains("]](../apache-poi-2) diff --git a/apache-poi/pom.xml b/apache-poi/pom.xml index e52f9e5ade23..1182b01b2cf0 100644 --- a/apache-poi/pom.xml +++ b/apache-poi/pom.xml @@ -69,12 +69,11 @@ - 5.3.0 + 5.5.0 1.0.9 - 0.18.0 + 0.19.0 3.3.1 2.23.1 - 2.22.2 4.2.0 diff --git a/apache-seata/README.md b/apache-seata/README.md new file mode 100644 index 000000000000..3e641e31e324 --- /dev/null +++ b/apache-seata/README.md @@ -0,0 +1,57 @@ +# Apache Seata Example Project + +This project shows an example of using Apache Seata for distributed transactions. + +## Service Structure + +This project represents 4 services: +* `apache-seata-shop-service` +* `apache-seata-inventory-service` +* `apache-seata-order-service` +* `apache-seata-billing-service` + +All of these work with a Postgres database, and additionally the Shop service makes API calls to the other three, +such that all of this is considered to be a single distributed transaction. + +## Starting The Project + +We can start the project using Docker Compose. + +```shell +$ docker compose up +``` + +This will start 6 containers: + +* Apache Seata +* Postgres +* `apache-seata-shop-service` +* `apache-seata-inventory-service` +* `apache-seata-order-service` +* `apache-seata-billing-service` + +Where `apache-seaa-shop-service` acts as the entrypoint into the application. + +# Testing + +We can make HTTP calls into the application by making POST calls to the `/shop/{mode}` endpoint of the `apache-seata-shop-service`. + +If the `{mode}` parameter is set to `shop`, `inventory`, `order` or `billing` then that service will fail during the transaction. +Anything else and the call will be successful. + +For example: +```shell +$ curl -X POST localhost:8080/shop/order +``` + +Will make a request that fails within the Order service. + +# Database Access + +We can access the database used by these services using Docker: + +```shell +$ docker exec -it apache-seata-postgres-1 psql --user seata seata +``` + +This opens a psql prompt inside the database, allowing us to explore the state of all the tables. \ No newline at end of file diff --git a/apache-seata/billing-service/.dockerignore b/apache-seata/billing-service/.dockerignore new file mode 100644 index 000000000000..eb5a316cbd19 --- /dev/null +++ b/apache-seata/billing-service/.dockerignore @@ -0,0 +1 @@ +target diff --git a/apache-seata/billing-service/.gitattributes b/apache-seata/billing-service/.gitattributes new file mode 100644 index 000000000000..3b41682ac579 --- /dev/null +++ b/apache-seata/billing-service/.gitattributes @@ -0,0 +1,2 @@ +/mvnw text eol=lf +*.cmd text eol=crlf diff --git a/apache-seata/billing-service/.gitignore b/apache-seata/billing-service/.gitignore new file mode 100644 index 000000000000..667aaef0c891 --- /dev/null +++ b/apache-seata/billing-service/.gitignore @@ -0,0 +1,33 @@ +HELP.md +target/ +.mvn/wrapper/maven-wrapper.jar +!**/src/main/**/target/ +!**/src/test/**/target/ + +### STS ### +.apt_generated +.classpath +.factorypath +.project +.settings +.springBeans +.sts4-cache + +### IntelliJ IDEA ### +.idea +*.iws +*.iml +*.ipr + +### NetBeans ### +/nbproject/private/ +/nbbuild/ +/dist/ +/nbdist/ +/.nb-gradle/ +build/ +!**/src/main/**/build/ +!**/src/test/**/build/ + +### VS Code ### +.vscode/ diff --git a/apache-seata/billing-service/.mvn/wrapper/maven-wrapper.properties b/apache-seata/billing-service/.mvn/wrapper/maven-wrapper.properties new file mode 100644 index 000000000000..8dea6c227c08 --- /dev/null +++ b/apache-seata/billing-service/.mvn/wrapper/maven-wrapper.properties @@ -0,0 +1,3 @@ +wrapperVersion=3.3.4 +distributionType=only-script +distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.12/apache-maven-3.9.12-bin.zip diff --git a/apache-seata/billing-service/Dockerfile b/apache-seata/billing-service/Dockerfile new file mode 100644 index 000000000000..258559ccad4d --- /dev/null +++ b/apache-seata/billing-service/Dockerfile @@ -0,0 +1,18 @@ +FROM maven:3.9.13-eclipse-temurin-17-alpine AS build + +WORKDIR /app + +COPY pom.xml . +RUN mvn compile + +COPY . . +RUN mvn clean package -DskipTests + + + +FROM eclipse-temurin:17 + +WORKDIR /app +COPY --from=build /app/target/*.jar app.jar + +ENTRYPOINT ["java", "-jar", "app.jar"] diff --git a/apache-seata/billing-service/mvnw b/apache-seata/billing-service/mvnw new file mode 100755 index 000000000000..bd8896bf2217 --- /dev/null +++ b/apache-seata/billing-service/mvnw @@ -0,0 +1,295 @@ +#!/bin/sh +# ---------------------------------------------------------------------------- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# ---------------------------------------------------------------------------- + +# ---------------------------------------------------------------------------- +# Apache Maven Wrapper startup batch script, version 3.3.4 +# +# Optional ENV vars +# ----------------- +# JAVA_HOME - location of a JDK home dir, required when download maven via java source +# MVNW_REPOURL - repo url base for downloading maven distribution +# MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +# MVNW_VERBOSE - true: enable verbose log; debug: trace the mvnw script; others: silence the output +# ---------------------------------------------------------------------------- + +set -euf +[ "${MVNW_VERBOSE-}" != debug ] || set -x + +# OS specific support. +native_path() { printf %s\\n "$1"; } +case "$(uname)" in +CYGWIN* | MINGW*) + [ -z "${JAVA_HOME-}" ] || JAVA_HOME="$(cygpath --unix "$JAVA_HOME")" + native_path() { cygpath --path --windows "$1"; } + ;; +esac + +# set JAVACMD and JAVACCMD +set_java_home() { + # For Cygwin and MinGW, ensure paths are in Unix format before anything is touched + if [ -n "${JAVA_HOME-}" ]; then + if [ -x "$JAVA_HOME/jre/sh/java" ]; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + JAVACCMD="$JAVA_HOME/jre/sh/javac" + else + JAVACMD="$JAVA_HOME/bin/java" + JAVACCMD="$JAVA_HOME/bin/javac" + + if [ ! -x "$JAVACMD" ] || [ ! -x "$JAVACCMD" ]; then + echo "The JAVA_HOME environment variable is not defined correctly, so mvnw cannot run." >&2 + echo "JAVA_HOME is set to \"$JAVA_HOME\", but \"\$JAVA_HOME/bin/java\" or \"\$JAVA_HOME/bin/javac\" does not exist." >&2 + return 1 + fi + fi + else + JAVACMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v java + )" || : + JAVACCMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v javac + )" || : + + if [ ! -x "${JAVACMD-}" ] || [ ! -x "${JAVACCMD-}" ]; then + echo "The java/javac command does not exist in PATH nor is JAVA_HOME set, so mvnw cannot run." >&2 + return 1 + fi + fi +} + +# hash string like Java String::hashCode +hash_string() { + str="${1:-}" h=0 + while [ -n "$str" ]; do + char="${str%"${str#?}"}" + h=$(((h * 31 + $(LC_CTYPE=C printf %d "'$char")) % 4294967296)) + str="${str#?}" + done + printf %x\\n $h +} + +verbose() { :; } +[ "${MVNW_VERBOSE-}" != true ] || verbose() { printf %s\\n "${1-}"; } + +die() { + printf %s\\n "$1" >&2 + exit 1 +} + +trim() { + # MWRAPPER-139: + # Trims trailing and leading whitespace, carriage returns, tabs, and linefeeds. + # Needed for removing poorly interpreted newline sequences when running in more + # exotic environments such as mingw bash on Windows. + printf "%s" "${1}" | tr -d '[:space:]' +} + +scriptDir="$(dirname "$0")" +scriptName="$(basename "$0")" + +# parse distributionUrl and optional distributionSha256Sum, requires .mvn/wrapper/maven-wrapper.properties +while IFS="=" read -r key value; do + case "${key-}" in + distributionUrl) distributionUrl=$(trim "${value-}") ;; + distributionSha256Sum) distributionSha256Sum=$(trim "${value-}") ;; + esac +done <"$scriptDir/.mvn/wrapper/maven-wrapper.properties" +[ -n "${distributionUrl-}" ] || die "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" + +case "${distributionUrl##*/}" in +maven-mvnd-*bin.*) + MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ + case "${PROCESSOR_ARCHITECTURE-}${PROCESSOR_ARCHITEW6432-}:$(uname -a)" in + *AMD64:CYGWIN* | *AMD64:MINGW*) distributionPlatform=windows-amd64 ;; + :Darwin*x86_64) distributionPlatform=darwin-amd64 ;; + :Darwin*arm64) distributionPlatform=darwin-aarch64 ;; + :Linux*x86_64*) distributionPlatform=linux-amd64 ;; + *) + echo "Cannot detect native platform for mvnd on $(uname)-$(uname -m), use pure java version" >&2 + distributionPlatform=linux-amd64 + ;; + esac + distributionUrl="${distributionUrl%-bin.*}-$distributionPlatform.zip" + ;; +maven-mvnd-*) MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ ;; +*) MVN_CMD="mvn${scriptName#mvnw}" _MVNW_REPO_PATTERN=/org/apache/maven/ ;; +esac + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +[ -z "${MVNW_REPOURL-}" ] || distributionUrl="$MVNW_REPOURL$_MVNW_REPO_PATTERN${distributionUrl#*"$_MVNW_REPO_PATTERN"}" +distributionUrlName="${distributionUrl##*/}" +distributionUrlNameMain="${distributionUrlName%.*}" +distributionUrlNameMain="${distributionUrlNameMain%-bin}" +MAVEN_USER_HOME="${MAVEN_USER_HOME:-${HOME}/.m2}" +MAVEN_HOME="${MAVEN_USER_HOME}/wrapper/dists/${distributionUrlNameMain-}/$(hash_string "$distributionUrl")" + +exec_maven() { + unset MVNW_VERBOSE MVNW_USERNAME MVNW_PASSWORD MVNW_REPOURL || : + exec "$MAVEN_HOME/bin/$MVN_CMD" "$@" || die "cannot exec $MAVEN_HOME/bin/$MVN_CMD" +} + +if [ -d "$MAVEN_HOME" ]; then + verbose "found existing MAVEN_HOME at $MAVEN_HOME" + exec_maven "$@" +fi + +case "${distributionUrl-}" in +*?-bin.zip | *?maven-mvnd-?*-?*.zip) ;; +*) die "distributionUrl is not valid, must match *-bin.zip or maven-mvnd-*.zip, but found '${distributionUrl-}'" ;; +esac + +# prepare tmp dir +if TMP_DOWNLOAD_DIR="$(mktemp -d)" && [ -d "$TMP_DOWNLOAD_DIR" ]; then + clean() { rm -rf -- "$TMP_DOWNLOAD_DIR"; } + trap clean HUP INT TERM EXIT +else + die "cannot create temp dir" +fi + +mkdir -p -- "${MAVEN_HOME%/*}" + +# Download and Install Apache Maven +verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +verbose "Downloading from: $distributionUrl" +verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +# select .zip or .tar.gz +if ! command -v unzip >/dev/null; then + distributionUrl="${distributionUrl%.zip}.tar.gz" + distributionUrlName="${distributionUrl##*/}" +fi + +# verbose opt +__MVNW_QUIET_WGET=--quiet __MVNW_QUIET_CURL=--silent __MVNW_QUIET_UNZIP=-q __MVNW_QUIET_TAR='' +[ "${MVNW_VERBOSE-}" != true ] || __MVNW_QUIET_WGET='' __MVNW_QUIET_CURL='' __MVNW_QUIET_UNZIP='' __MVNW_QUIET_TAR=v + +# normalize http auth +case "${MVNW_PASSWORD:+has-password}" in +'') MVNW_USERNAME='' MVNW_PASSWORD='' ;; +has-password) [ -n "${MVNW_USERNAME-}" ] || MVNW_USERNAME='' MVNW_PASSWORD='' ;; +esac + +if [ -z "${MVNW_USERNAME-}" ] && command -v wget >/dev/null; then + verbose "Found wget ... using wget" + wget ${__MVNW_QUIET_WGET:+"$__MVNW_QUIET_WGET"} "$distributionUrl" -O "$TMP_DOWNLOAD_DIR/$distributionUrlName" || die "wget: Failed to fetch $distributionUrl" +elif [ -z "${MVNW_USERNAME-}" ] && command -v curl >/dev/null; then + verbose "Found curl ... using curl" + curl ${__MVNW_QUIET_CURL:+"$__MVNW_QUIET_CURL"} -f -L -o "$TMP_DOWNLOAD_DIR/$distributionUrlName" "$distributionUrl" || die "curl: Failed to fetch $distributionUrl" +elif set_java_home; then + verbose "Falling back to use Java to download" + javaSource="$TMP_DOWNLOAD_DIR/Downloader.java" + targetZip="$TMP_DOWNLOAD_DIR/$distributionUrlName" + cat >"$javaSource" <<-END + public class Downloader extends java.net.Authenticator + { + protected java.net.PasswordAuthentication getPasswordAuthentication() + { + return new java.net.PasswordAuthentication( System.getenv( "MVNW_USERNAME" ), System.getenv( "MVNW_PASSWORD" ).toCharArray() ); + } + public static void main( String[] args ) throws Exception + { + setDefault( new Downloader() ); + java.nio.file.Files.copy( java.net.URI.create( args[0] ).toURL().openStream(), java.nio.file.Paths.get( args[1] ).toAbsolutePath().normalize() ); + } + } + END + # For Cygwin/MinGW, switch paths to Windows format before running javac and java + verbose " - Compiling Downloader.java ..." + "$(native_path "$JAVACCMD")" "$(native_path "$javaSource")" || die "Failed to compile Downloader.java" + verbose " - Running Downloader.java ..." + "$(native_path "$JAVACMD")" -cp "$(native_path "$TMP_DOWNLOAD_DIR")" Downloader "$distributionUrl" "$(native_path "$targetZip")" +fi + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +if [ -n "${distributionSha256Sum-}" ]; then + distributionSha256Result=false + if [ "$MVN_CMD" = mvnd.sh ]; then + echo "Checksum validation is not supported for maven-mvnd." >&2 + echo "Please disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + elif command -v sha256sum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | sha256sum -c - >/dev/null 2>&1; then + distributionSha256Result=true + fi + elif command -v shasum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | shasum -a 256 -c >/dev/null 2>&1; then + distributionSha256Result=true + fi + else + echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available." >&2 + echo "Please install either command, or disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + fi + if [ $distributionSha256Result = false ]; then + echo "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised." >&2 + echo "If you updated your Maven version, you need to update the specified distributionSha256Sum property." >&2 + exit 1 + fi +fi + +# unzip and move +if command -v unzip >/dev/null; then + unzip ${__MVNW_QUIET_UNZIP:+"$__MVNW_QUIET_UNZIP"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -d "$TMP_DOWNLOAD_DIR" || die "failed to unzip" +else + tar xzf${__MVNW_QUIET_TAR:+"$__MVNW_QUIET_TAR"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -C "$TMP_DOWNLOAD_DIR" || die "failed to untar" +fi + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +actualDistributionDir="" + +# First try the expected directory name (for regular distributions) +if [ -d "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" ]; then + if [ -f "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain/bin/$MVN_CMD" ]; then + actualDistributionDir="$distributionUrlNameMain" + fi +fi + +# If not found, search for any directory with the Maven executable (for snapshots) +if [ -z "$actualDistributionDir" ]; then + # enable globbing to iterate over items + set +f + for dir in "$TMP_DOWNLOAD_DIR"/*; do + if [ -d "$dir" ]; then + if [ -f "$dir/bin/$MVN_CMD" ]; then + actualDistributionDir="$(basename "$dir")" + break + fi + fi + done + set -f +fi + +if [ -z "$actualDistributionDir" ]; then + verbose "Contents of $TMP_DOWNLOAD_DIR:" + verbose "$(ls -la "$TMP_DOWNLOAD_DIR")" + die "Could not find Maven distribution directory in extracted archive" +fi + +verbose "Found extracted Maven distribution directory: $actualDistributionDir" +printf %s\\n "$distributionUrl" >"$TMP_DOWNLOAD_DIR/$actualDistributionDir/mvnw.url" +mv -- "$TMP_DOWNLOAD_DIR/$actualDistributionDir" "$MAVEN_HOME" || [ -d "$MAVEN_HOME" ] || die "fail to move MAVEN_HOME" + +clean || : +exec_maven "$@" diff --git a/apache-seata/billing-service/mvnw.cmd b/apache-seata/billing-service/mvnw.cmd new file mode 100644 index 000000000000..92450f932734 --- /dev/null +++ b/apache-seata/billing-service/mvnw.cmd @@ -0,0 +1,189 @@ +<# : batch portion +@REM ---------------------------------------------------------------------------- +@REM Licensed to the Apache Software Foundation (ASF) under one +@REM or more contributor license agreements. See the NOTICE file +@REM distributed with this work for additional information +@REM regarding copyright ownership. The ASF licenses this file +@REM to you under the Apache License, Version 2.0 (the +@REM "License"); you may not use this file except in compliance +@REM with the License. You may obtain a copy of the License at +@REM +@REM http://www.apache.org/licenses/LICENSE-2.0 +@REM +@REM Unless required by applicable law or agreed to in writing, +@REM software distributed under the License is distributed on an +@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +@REM KIND, either express or implied. See the License for the +@REM specific language governing permissions and limitations +@REM under the License. +@REM ---------------------------------------------------------------------------- + +@REM ---------------------------------------------------------------------------- +@REM Apache Maven Wrapper startup batch script, version 3.3.4 +@REM +@REM Optional ENV vars +@REM MVNW_REPOURL - repo url base for downloading maven distribution +@REM MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +@REM MVNW_VERBOSE - true: enable verbose log; others: silence the output +@REM ---------------------------------------------------------------------------- + +@IF "%__MVNW_ARG0_NAME__%"=="" (SET __MVNW_ARG0_NAME__=%~nx0) +@SET __MVNW_CMD__= +@SET __MVNW_ERROR__= +@SET __MVNW_PSMODULEP_SAVE=%PSModulePath% +@SET PSModulePath= +@FOR /F "usebackq tokens=1* delims==" %%A IN (`powershell -noprofile "& {$scriptDir='%~dp0'; $script='%__MVNW_ARG0_NAME__%'; icm -ScriptBlock ([Scriptblock]::Create((Get-Content -Raw '%~f0'))) -NoNewScope}"`) DO @( + IF "%%A"=="MVN_CMD" (set __MVNW_CMD__=%%B) ELSE IF "%%B"=="" (echo %%A) ELSE (echo %%A=%%B) +) +@SET PSModulePath=%__MVNW_PSMODULEP_SAVE% +@SET __MVNW_PSMODULEP_SAVE= +@SET __MVNW_ARG0_NAME__= +@SET MVNW_USERNAME= +@SET MVNW_PASSWORD= +@IF NOT "%__MVNW_CMD__%"=="" ("%__MVNW_CMD__%" %*) +@echo Cannot start maven from wrapper >&2 && exit /b 1 +@GOTO :EOF +: end batch / begin powershell #> + +$ErrorActionPreference = "Stop" +if ($env:MVNW_VERBOSE -eq "true") { + $VerbosePreference = "Continue" +} + +# calculate distributionUrl, requires .mvn/wrapper/maven-wrapper.properties +$distributionUrl = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionUrl +if (!$distributionUrl) { + Write-Error "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" +} + +switch -wildcard -casesensitive ( $($distributionUrl -replace '^.*/','') ) { + "maven-mvnd-*" { + $USE_MVND = $true + $distributionUrl = $distributionUrl -replace '-bin\.[^.]*$',"-windows-amd64.zip" + $MVN_CMD = "mvnd.cmd" + break + } + default { + $USE_MVND = $false + $MVN_CMD = $script -replace '^mvnw','mvn' + break + } +} + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +if ($env:MVNW_REPOURL) { + $MVNW_REPO_PATTERN = if ($USE_MVND -eq $False) { "/org/apache/maven/" } else { "/maven/mvnd/" } + $distributionUrl = "$env:MVNW_REPOURL$MVNW_REPO_PATTERN$($distributionUrl -replace "^.*$MVNW_REPO_PATTERN",'')" +} +$distributionUrlName = $distributionUrl -replace '^.*/','' +$distributionUrlNameMain = $distributionUrlName -replace '\.[^.]*$','' -replace '-bin$','' + +$MAVEN_M2_PATH = "$HOME/.m2" +if ($env:MAVEN_USER_HOME) { + $MAVEN_M2_PATH = "$env:MAVEN_USER_HOME" +} + +if (-not (Test-Path -Path $MAVEN_M2_PATH)) { + New-Item -Path $MAVEN_M2_PATH -ItemType Directory | Out-Null +} + +$MAVEN_WRAPPER_DISTS = $null +if ((Get-Item $MAVEN_M2_PATH).Target[0] -eq $null) { + $MAVEN_WRAPPER_DISTS = "$MAVEN_M2_PATH/wrapper/dists" +} else { + $MAVEN_WRAPPER_DISTS = (Get-Item $MAVEN_M2_PATH).Target[0] + "/wrapper/dists" +} + +$MAVEN_HOME_PARENT = "$MAVEN_WRAPPER_DISTS/$distributionUrlNameMain" +$MAVEN_HOME_NAME = ([System.Security.Cryptography.SHA256]::Create().ComputeHash([byte[]][char[]]$distributionUrl) | ForEach-Object {$_.ToString("x2")}) -join '' +$MAVEN_HOME = "$MAVEN_HOME_PARENT/$MAVEN_HOME_NAME" + +if (Test-Path -Path "$MAVEN_HOME" -PathType Container) { + Write-Verbose "found existing MAVEN_HOME at $MAVEN_HOME" + Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" + exit $? +} + +if (! $distributionUrlNameMain -or ($distributionUrlName -eq $distributionUrlNameMain)) { + Write-Error "distributionUrl is not valid, must end with *-bin.zip, but found $distributionUrl" +} + +# prepare tmp dir +$TMP_DOWNLOAD_DIR_HOLDER = New-TemporaryFile +$TMP_DOWNLOAD_DIR = New-Item -Itemtype Directory -Path "$TMP_DOWNLOAD_DIR_HOLDER.dir" +$TMP_DOWNLOAD_DIR_HOLDER.Delete() | Out-Null +trap { + if ($TMP_DOWNLOAD_DIR.Exists) { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } + } +} + +New-Item -Itemtype Directory -Path "$MAVEN_HOME_PARENT" -Force | Out-Null + +# Download and Install Apache Maven +Write-Verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +Write-Verbose "Downloading from: $distributionUrl" +Write-Verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +$webclient = New-Object System.Net.WebClient +if ($env:MVNW_USERNAME -and $env:MVNW_PASSWORD) { + $webclient.Credentials = New-Object System.Net.NetworkCredential($env:MVNW_USERNAME, $env:MVNW_PASSWORD) +} +[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 +$webclient.DownloadFile($distributionUrl, "$TMP_DOWNLOAD_DIR/$distributionUrlName") | Out-Null + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +$distributionSha256Sum = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionSha256Sum +if ($distributionSha256Sum) { + if ($USE_MVND) { + Write-Error "Checksum validation is not supported for maven-mvnd. `nPlease disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." + } + Import-Module $PSHOME\Modules\Microsoft.PowerShell.Utility -Function Get-FileHash + if ((Get-FileHash "$TMP_DOWNLOAD_DIR/$distributionUrlName" -Algorithm SHA256).Hash.ToLower() -ne $distributionSha256Sum) { + Write-Error "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised. If you updated your Maven version, you need to update the specified distributionSha256Sum property." + } +} + +# unzip and move +Expand-Archive "$TMP_DOWNLOAD_DIR/$distributionUrlName" -DestinationPath "$TMP_DOWNLOAD_DIR" | Out-Null + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +$actualDistributionDir = "" + +# First try the expected directory name (for regular distributions) +$expectedPath = Join-Path "$TMP_DOWNLOAD_DIR" "$distributionUrlNameMain" +$expectedMvnPath = Join-Path "$expectedPath" "bin/$MVN_CMD" +if ((Test-Path -Path $expectedPath -PathType Container) -and (Test-Path -Path $expectedMvnPath -PathType Leaf)) { + $actualDistributionDir = $distributionUrlNameMain +} + +# If not found, search for any directory with the Maven executable (for snapshots) +if (!$actualDistributionDir) { + Get-ChildItem -Path "$TMP_DOWNLOAD_DIR" -Directory | ForEach-Object { + $testPath = Join-Path $_.FullName "bin/$MVN_CMD" + if (Test-Path -Path $testPath -PathType Leaf) { + $actualDistributionDir = $_.Name + } + } +} + +if (!$actualDistributionDir) { + Write-Error "Could not find Maven distribution directory in extracted archive" +} + +Write-Verbose "Found extracted Maven distribution directory: $actualDistributionDir" +Rename-Item -Path "$TMP_DOWNLOAD_DIR/$actualDistributionDir" -NewName $MAVEN_HOME_NAME | Out-Null +try { + Move-Item -Path "$TMP_DOWNLOAD_DIR/$MAVEN_HOME_NAME" -Destination $MAVEN_HOME_PARENT | Out-Null +} catch { + if (! (Test-Path -Path "$MAVEN_HOME" -PathType Container)) { + Write-Error "fail to move MAVEN_HOME" + } +} finally { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } +} + +Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" diff --git a/apache-seata/billing-service/pom.xml b/apache-seata/billing-service/pom.xml new file mode 100644 index 000000000000..3521c1966564 --- /dev/null +++ b/apache-seata/billing-service/pom.xml @@ -0,0 +1,59 @@ + + + 4.0.0 + com.baeldung + apache-seata-billing-service + 1.0.0-SNAPSHOT + apache-seata-billing-service + Apache Seata - Billing Service + + + org.springframework.boot + spring-boot-starter-parent + 3.5.11 + + + + + + org.springframework.boot + spring-boot-starter-jdbc + + + org.springframework.boot + spring-boot-starter-web + + + org.apache.seata + seata-spring-boot-starter + ${seata.version} + + + org.postgresql + postgresql + runtime + + + org.springframework.boot + spring-boot-starter-test + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + + + 17 + 2.6.0 + + + diff --git a/apache-seata/billing-service/src/main/java/com/baeldung/billing/ApacheSeataBillingApplication.java b/apache-seata/billing-service/src/main/java/com/baeldung/billing/ApacheSeataBillingApplication.java new file mode 100644 index 000000000000..8ccb929a630a --- /dev/null +++ b/apache-seata/billing-service/src/main/java/com/baeldung/billing/ApacheSeataBillingApplication.java @@ -0,0 +1,13 @@ +package com.baeldung.billing; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +@SpringBootApplication +public class ApacheSeataBillingApplication { + + public static void main(String[] args) { + SpringApplication.run(ApacheSeataBillingApplication.class, args); + } + +} diff --git a/apache-seata/billing-service/src/main/java/com/baeldung/billing/Controller.java b/apache-seata/billing-service/src/main/java/com/baeldung/billing/Controller.java new file mode 100644 index 000000000000..5762c0e785e0 --- /dev/null +++ b/apache-seata/billing-service/src/main/java/com/baeldung/billing/Controller.java @@ -0,0 +1,23 @@ +package com.baeldung.billing; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.transaction.annotation.Transactional; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RestController; + +@RestController +public class Controller { + @Autowired + private Repository repository; + + @PostMapping("/billing/{mode}") + @Transactional + public void handle(@PathVariable("mode") String mode) { + repository.updateDatabase(); + + if ("billing".equals(mode)) { + throw new RuntimeException("Billing Service failed"); + } + } +} diff --git a/apache-seata/billing-service/src/main/java/com/baeldung/billing/Repository.java b/apache-seata/billing-service/src/main/java/com/baeldung/billing/Repository.java new file mode 100644 index 000000000000..ad1171000dcc --- /dev/null +++ b/apache-seata/billing-service/src/main/java/com/baeldung/billing/Repository.java @@ -0,0 +1,34 @@ +package com.baeldung.billing; + +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Map; +import java.util.UUID; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; +import org.springframework.stereotype.Service; + +@Service +public class Repository { + private static final Logger LOG = LoggerFactory.getLogger(Repository.class); + + @Autowired + private NamedParameterJdbcTemplate jdbcTemplate; + + public void updateDatabase() { + var params = Map.of( + "id", UUID.randomUUID().toString(), + "created", Instant.now().atOffset(ZoneOffset.UTC) + ); + + LOG.info("Updating database with {}", params); + + int result = jdbcTemplate.update("INSERT INTO billing_table(id, created) VALUES (:id, :created)", + params); + + LOG.info("Updating database with result {}", result); + } +} diff --git a/apache-seata/billing-service/src/main/java/com/baeldung/billing/SeataXidFilter.java b/apache-seata/billing-service/src/main/java/com/baeldung/billing/SeataXidFilter.java new file mode 100644 index 000000000000..f7836abf74b2 --- /dev/null +++ b/apache-seata/billing-service/src/main/java/com/baeldung/billing/SeataXidFilter.java @@ -0,0 +1,50 @@ +package com.baeldung.billing; + +import java.io.IOException; + +import org.apache.seata.core.context.RootContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.core.Ordered; +import org.springframework.core.annotation.Order; +import org.springframework.stereotype.Component; +import org.springframework.util.StringUtils; + +import jakarta.servlet.Filter; +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.ServletRequest; +import jakarta.servlet.ServletResponse; +import jakarta.servlet.http.HttpServletRequest; + +@Component +@Order(Ordered.HIGHEST_PRECEDENCE) +public class SeataXidFilter implements Filter { + private static final Logger LOG = LoggerFactory.getLogger(SeataXidFilter.class); + + @Override + public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) + throws IOException, ServletException { + + HttpServletRequest httpRequest = (HttpServletRequest) req; + String xid = httpRequest.getHeader(RootContext.KEY_XID); + + boolean bound = false; + if (StringUtils.hasText(xid) && !xid.equals(RootContext.getXID())) { + LOG.info("Receiving Seata XID: {}", xid); + + RootContext.bind(xid); + bound = true; + } + + try { + chain.doFilter(req, res); + } finally { + // Always unbind — leaking an XID into the next request on this thread + // is a subtle bug that causes phantom branch enrollments. + if (bound) { + RootContext.unbind(); + } + } + } +} \ No newline at end of file diff --git a/apache-seata/billing-service/src/main/resources/application.properties b/apache-seata/billing-service/src/main/resources/application.properties new file mode 100644 index 000000000000..a4baed5fe0ee --- /dev/null +++ b/apache-seata/billing-service/src/main/resources/application.properties @@ -0,0 +1,25 @@ +spring.application.name=apache-seata-c + +spring.datasource.generate-unique-name=false +spring.datasource.url=jdbc:postgresql://postgres:5432/seata +spring.datasource.username=seata +spring.datasource.password=seata +spring.datasource.driver-class-name=org.postgresql.Driver + +server.port=8083 + +seata.enabled=true +seata.application-id=${spring.application.name} +seata.tx-service-group=my_tx_group + +seata.registry.type=file +seata.registry.file.name=seata.conf + +seata.config.type=file +seata.config.file.name=seata.conf + +seata.service.vgroup-mapping.my_tx_group=default +seata.service.grouplist.default=seata-server:8091 + +seata.data-source-proxy-mode=AT +seata.enable-auto-data-source-proxy=true diff --git a/apache-seata/billing-service/src/main/resources/seata.conf b/apache-seata/billing-service/src/main/resources/seata.conf new file mode 100644 index 000000000000..0219477600d9 --- /dev/null +++ b/apache-seata/billing-service/src/main/resources/seata.conf @@ -0,0 +1,60 @@ +transport { + type = "TCP" + server = "NIO" + heartbeat = true + thread-factory { + boss-thread-prefix = "NettyBoss" + worker-thread-prefix = "NettyServerNIOWorker" + server-executor-thread-size = 100 + share-boss-worker = false + client-selector-thread-size = 1 + client-selector-thread-prefix = "NettyClientSelector" + client-worker-thread-prefix = "NettyClientWorkerThread" + } + shutdown { + wait = 3 + } + serialization = "seata" + compressor = "none" +} + +service { + vgroupMapping.my_tx_group = "default" + default.grouplist = "seata-server:8091" + enableDegrade = false + disableGlobalTransaction = false +} + +client { + rm { + asyncCommitBufferLimit = 10000 + lock { + retryInterval = 10 + retryTimes = 30 + retryPolicyBranchRollbackOnConflict = true + } + reportRetryCount = 5 + tableMetaCheckEnable = false + reportSuccessEnable = false + sagaBranchRegisterEnable = false + } + tm { + commitRetryCount = 5 + rollbackRetryCount = 5 + defaultGlobalTransactionTimeout = 60000 + degradeCheck = false + } + undo { + dataValidation = true + logSerialization = "jackson" + logTable = "undo_log" + compress { + enable = true + type = "zip" + threshold = "64k" + } + } + log { + exceptionRate = 100 + } +} \ No newline at end of file diff --git a/apache-seata/docker-compose.yml b/apache-seata/docker-compose.yml new file mode 100644 index 000000000000..090f154b412d --- /dev/null +++ b/apache-seata/docker-compose.yml @@ -0,0 +1,77 @@ +services: + seata-server: + image: apache/seata-server:2.6.0 + + postgres: + image: postgres + environment: + POSTGRES_DB: seata + POSTGRES_USER: seata + POSTGRES_PASSWORD: seata + volumes: + - ./sql:/docker-entrypoint-initdb.d + + shop-service: + platform: linux/amd64 + build: + context: shop-service + environment: + SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/seata + SPRING_DATASOURCE_USERNAME: seata + SPRING_DATASOURCE_PASSWORD: seata + INVENTORY_SERVICE_URL: http://inventory-service:8081 + ORDER_SERVICE_URL: http://order-service:8082 + BILLING_SERVICE_URL: http://billing-service:8083 + ports: + - 127.0.0.1:8080:8080 + links: + - postgres + - inventory-service + - order-service + - billing-service + depends_on: + - postgres + - seata-server + + inventory-service: + platform: linux/amd64 + build: + context: inventory-service + environment: + SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/seata + SPRING_DATASOURCE_USERNAME: seata + SPRING_DATASOURCE_PASSWORD: seata + links: + - postgres + depends_on: + - postgres + - seata-server + + order-service: + platform: linux/amd64 + build: + context: order-service + environment: + SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/seata + SPRING_DATASOURCE_USERNAME: seata + SPRING_DATASOURCE_PASSWORD: seata + links: + - postgres + depends_on: + - postgres + - seata-server + + billing-service: + platform: linux/amd64 + build: + context: billing-service + environment: + SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/seata + SPRING_DATASOURCE_USERNAME: seata + SPRING_DATASOURCE_PASSWORD: seata + links: + - postgres + depends_on: + - postgres + - seata-server + diff --git a/apache-seata/inventory-service/.dockerignore b/apache-seata/inventory-service/.dockerignore new file mode 100644 index 000000000000..eb5a316cbd19 --- /dev/null +++ b/apache-seata/inventory-service/.dockerignore @@ -0,0 +1 @@ +target diff --git a/apache-seata/inventory-service/.gitattributes b/apache-seata/inventory-service/.gitattributes new file mode 100644 index 000000000000..3b41682ac579 --- /dev/null +++ b/apache-seata/inventory-service/.gitattributes @@ -0,0 +1,2 @@ +/mvnw text eol=lf +*.cmd text eol=crlf diff --git a/apache-seata/inventory-service/.gitignore b/apache-seata/inventory-service/.gitignore new file mode 100644 index 000000000000..667aaef0c891 --- /dev/null +++ b/apache-seata/inventory-service/.gitignore @@ -0,0 +1,33 @@ +HELP.md +target/ +.mvn/wrapper/maven-wrapper.jar +!**/src/main/**/target/ +!**/src/test/**/target/ + +### STS ### +.apt_generated +.classpath +.factorypath +.project +.settings +.springBeans +.sts4-cache + +### IntelliJ IDEA ### +.idea +*.iws +*.iml +*.ipr + +### NetBeans ### +/nbproject/private/ +/nbbuild/ +/dist/ +/nbdist/ +/.nb-gradle/ +build/ +!**/src/main/**/build/ +!**/src/test/**/build/ + +### VS Code ### +.vscode/ diff --git a/apache-seata/inventory-service/.mvn/wrapper/maven-wrapper.properties b/apache-seata/inventory-service/.mvn/wrapper/maven-wrapper.properties new file mode 100644 index 000000000000..8dea6c227c08 --- /dev/null +++ b/apache-seata/inventory-service/.mvn/wrapper/maven-wrapper.properties @@ -0,0 +1,3 @@ +wrapperVersion=3.3.4 +distributionType=only-script +distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.12/apache-maven-3.9.12-bin.zip diff --git a/apache-seata/inventory-service/Dockerfile b/apache-seata/inventory-service/Dockerfile new file mode 100644 index 000000000000..258559ccad4d --- /dev/null +++ b/apache-seata/inventory-service/Dockerfile @@ -0,0 +1,18 @@ +FROM maven:3.9.13-eclipse-temurin-17-alpine AS build + +WORKDIR /app + +COPY pom.xml . +RUN mvn compile + +COPY . . +RUN mvn clean package -DskipTests + + + +FROM eclipse-temurin:17 + +WORKDIR /app +COPY --from=build /app/target/*.jar app.jar + +ENTRYPOINT ["java", "-jar", "app.jar"] diff --git a/apache-seata/inventory-service/mvnw b/apache-seata/inventory-service/mvnw new file mode 100755 index 000000000000..bd8896bf2217 --- /dev/null +++ b/apache-seata/inventory-service/mvnw @@ -0,0 +1,295 @@ +#!/bin/sh +# ---------------------------------------------------------------------------- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# ---------------------------------------------------------------------------- + +# ---------------------------------------------------------------------------- +# Apache Maven Wrapper startup batch script, version 3.3.4 +# +# Optional ENV vars +# ----------------- +# JAVA_HOME - location of a JDK home dir, required when download maven via java source +# MVNW_REPOURL - repo url base for downloading maven distribution +# MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +# MVNW_VERBOSE - true: enable verbose log; debug: trace the mvnw script; others: silence the output +# ---------------------------------------------------------------------------- + +set -euf +[ "${MVNW_VERBOSE-}" != debug ] || set -x + +# OS specific support. +native_path() { printf %s\\n "$1"; } +case "$(uname)" in +CYGWIN* | MINGW*) + [ -z "${JAVA_HOME-}" ] || JAVA_HOME="$(cygpath --unix "$JAVA_HOME")" + native_path() { cygpath --path --windows "$1"; } + ;; +esac + +# set JAVACMD and JAVACCMD +set_java_home() { + # For Cygwin and MinGW, ensure paths are in Unix format before anything is touched + if [ -n "${JAVA_HOME-}" ]; then + if [ -x "$JAVA_HOME/jre/sh/java" ]; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + JAVACCMD="$JAVA_HOME/jre/sh/javac" + else + JAVACMD="$JAVA_HOME/bin/java" + JAVACCMD="$JAVA_HOME/bin/javac" + + if [ ! -x "$JAVACMD" ] || [ ! -x "$JAVACCMD" ]; then + echo "The JAVA_HOME environment variable is not defined correctly, so mvnw cannot run." >&2 + echo "JAVA_HOME is set to \"$JAVA_HOME\", but \"\$JAVA_HOME/bin/java\" or \"\$JAVA_HOME/bin/javac\" does not exist." >&2 + return 1 + fi + fi + else + JAVACMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v java + )" || : + JAVACCMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v javac + )" || : + + if [ ! -x "${JAVACMD-}" ] || [ ! -x "${JAVACCMD-}" ]; then + echo "The java/javac command does not exist in PATH nor is JAVA_HOME set, so mvnw cannot run." >&2 + return 1 + fi + fi +} + +# hash string like Java String::hashCode +hash_string() { + str="${1:-}" h=0 + while [ -n "$str" ]; do + char="${str%"${str#?}"}" + h=$(((h * 31 + $(LC_CTYPE=C printf %d "'$char")) % 4294967296)) + str="${str#?}" + done + printf %x\\n $h +} + +verbose() { :; } +[ "${MVNW_VERBOSE-}" != true ] || verbose() { printf %s\\n "${1-}"; } + +die() { + printf %s\\n "$1" >&2 + exit 1 +} + +trim() { + # MWRAPPER-139: + # Trims trailing and leading whitespace, carriage returns, tabs, and linefeeds. + # Needed for removing poorly interpreted newline sequences when running in more + # exotic environments such as mingw bash on Windows. + printf "%s" "${1}" | tr -d '[:space:]' +} + +scriptDir="$(dirname "$0")" +scriptName="$(basename "$0")" + +# parse distributionUrl and optional distributionSha256Sum, requires .mvn/wrapper/maven-wrapper.properties +while IFS="=" read -r key value; do + case "${key-}" in + distributionUrl) distributionUrl=$(trim "${value-}") ;; + distributionSha256Sum) distributionSha256Sum=$(trim "${value-}") ;; + esac +done <"$scriptDir/.mvn/wrapper/maven-wrapper.properties" +[ -n "${distributionUrl-}" ] || die "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" + +case "${distributionUrl##*/}" in +maven-mvnd-*bin.*) + MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ + case "${PROCESSOR_ARCHITECTURE-}${PROCESSOR_ARCHITEW6432-}:$(uname -a)" in + *AMD64:CYGWIN* | *AMD64:MINGW*) distributionPlatform=windows-amd64 ;; + :Darwin*x86_64) distributionPlatform=darwin-amd64 ;; + :Darwin*arm64) distributionPlatform=darwin-aarch64 ;; + :Linux*x86_64*) distributionPlatform=linux-amd64 ;; + *) + echo "Cannot detect native platform for mvnd on $(uname)-$(uname -m), use pure java version" >&2 + distributionPlatform=linux-amd64 + ;; + esac + distributionUrl="${distributionUrl%-bin.*}-$distributionPlatform.zip" + ;; +maven-mvnd-*) MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ ;; +*) MVN_CMD="mvn${scriptName#mvnw}" _MVNW_REPO_PATTERN=/org/apache/maven/ ;; +esac + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +[ -z "${MVNW_REPOURL-}" ] || distributionUrl="$MVNW_REPOURL$_MVNW_REPO_PATTERN${distributionUrl#*"$_MVNW_REPO_PATTERN"}" +distributionUrlName="${distributionUrl##*/}" +distributionUrlNameMain="${distributionUrlName%.*}" +distributionUrlNameMain="${distributionUrlNameMain%-bin}" +MAVEN_USER_HOME="${MAVEN_USER_HOME:-${HOME}/.m2}" +MAVEN_HOME="${MAVEN_USER_HOME}/wrapper/dists/${distributionUrlNameMain-}/$(hash_string "$distributionUrl")" + +exec_maven() { + unset MVNW_VERBOSE MVNW_USERNAME MVNW_PASSWORD MVNW_REPOURL || : + exec "$MAVEN_HOME/bin/$MVN_CMD" "$@" || die "cannot exec $MAVEN_HOME/bin/$MVN_CMD" +} + +if [ -d "$MAVEN_HOME" ]; then + verbose "found existing MAVEN_HOME at $MAVEN_HOME" + exec_maven "$@" +fi + +case "${distributionUrl-}" in +*?-bin.zip | *?maven-mvnd-?*-?*.zip) ;; +*) die "distributionUrl is not valid, must match *-bin.zip or maven-mvnd-*.zip, but found '${distributionUrl-}'" ;; +esac + +# prepare tmp dir +if TMP_DOWNLOAD_DIR="$(mktemp -d)" && [ -d "$TMP_DOWNLOAD_DIR" ]; then + clean() { rm -rf -- "$TMP_DOWNLOAD_DIR"; } + trap clean HUP INT TERM EXIT +else + die "cannot create temp dir" +fi + +mkdir -p -- "${MAVEN_HOME%/*}" + +# Download and Install Apache Maven +verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +verbose "Downloading from: $distributionUrl" +verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +# select .zip or .tar.gz +if ! command -v unzip >/dev/null; then + distributionUrl="${distributionUrl%.zip}.tar.gz" + distributionUrlName="${distributionUrl##*/}" +fi + +# verbose opt +__MVNW_QUIET_WGET=--quiet __MVNW_QUIET_CURL=--silent __MVNW_QUIET_UNZIP=-q __MVNW_QUIET_TAR='' +[ "${MVNW_VERBOSE-}" != true ] || __MVNW_QUIET_WGET='' __MVNW_QUIET_CURL='' __MVNW_QUIET_UNZIP='' __MVNW_QUIET_TAR=v + +# normalize http auth +case "${MVNW_PASSWORD:+has-password}" in +'') MVNW_USERNAME='' MVNW_PASSWORD='' ;; +has-password) [ -n "${MVNW_USERNAME-}" ] || MVNW_USERNAME='' MVNW_PASSWORD='' ;; +esac + +if [ -z "${MVNW_USERNAME-}" ] && command -v wget >/dev/null; then + verbose "Found wget ... using wget" + wget ${__MVNW_QUIET_WGET:+"$__MVNW_QUIET_WGET"} "$distributionUrl" -O "$TMP_DOWNLOAD_DIR/$distributionUrlName" || die "wget: Failed to fetch $distributionUrl" +elif [ -z "${MVNW_USERNAME-}" ] && command -v curl >/dev/null; then + verbose "Found curl ... using curl" + curl ${__MVNW_QUIET_CURL:+"$__MVNW_QUIET_CURL"} -f -L -o "$TMP_DOWNLOAD_DIR/$distributionUrlName" "$distributionUrl" || die "curl: Failed to fetch $distributionUrl" +elif set_java_home; then + verbose "Falling back to use Java to download" + javaSource="$TMP_DOWNLOAD_DIR/Downloader.java" + targetZip="$TMP_DOWNLOAD_DIR/$distributionUrlName" + cat >"$javaSource" <<-END + public class Downloader extends java.net.Authenticator + { + protected java.net.PasswordAuthentication getPasswordAuthentication() + { + return new java.net.PasswordAuthentication( System.getenv( "MVNW_USERNAME" ), System.getenv( "MVNW_PASSWORD" ).toCharArray() ); + } + public static void main( String[] args ) throws Exception + { + setDefault( new Downloader() ); + java.nio.file.Files.copy( java.net.URI.create( args[0] ).toURL().openStream(), java.nio.file.Paths.get( args[1] ).toAbsolutePath().normalize() ); + } + } + END + # For Cygwin/MinGW, switch paths to Windows format before running javac and java + verbose " - Compiling Downloader.java ..." + "$(native_path "$JAVACCMD")" "$(native_path "$javaSource")" || die "Failed to compile Downloader.java" + verbose " - Running Downloader.java ..." + "$(native_path "$JAVACMD")" -cp "$(native_path "$TMP_DOWNLOAD_DIR")" Downloader "$distributionUrl" "$(native_path "$targetZip")" +fi + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +if [ -n "${distributionSha256Sum-}" ]; then + distributionSha256Result=false + if [ "$MVN_CMD" = mvnd.sh ]; then + echo "Checksum validation is not supported for maven-mvnd." >&2 + echo "Please disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + elif command -v sha256sum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | sha256sum -c - >/dev/null 2>&1; then + distributionSha256Result=true + fi + elif command -v shasum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | shasum -a 256 -c >/dev/null 2>&1; then + distributionSha256Result=true + fi + else + echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available." >&2 + echo "Please install either command, or disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + fi + if [ $distributionSha256Result = false ]; then + echo "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised." >&2 + echo "If you updated your Maven version, you need to update the specified distributionSha256Sum property." >&2 + exit 1 + fi +fi + +# unzip and move +if command -v unzip >/dev/null; then + unzip ${__MVNW_QUIET_UNZIP:+"$__MVNW_QUIET_UNZIP"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -d "$TMP_DOWNLOAD_DIR" || die "failed to unzip" +else + tar xzf${__MVNW_QUIET_TAR:+"$__MVNW_QUIET_TAR"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -C "$TMP_DOWNLOAD_DIR" || die "failed to untar" +fi + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +actualDistributionDir="" + +# First try the expected directory name (for regular distributions) +if [ -d "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" ]; then + if [ -f "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain/bin/$MVN_CMD" ]; then + actualDistributionDir="$distributionUrlNameMain" + fi +fi + +# If not found, search for any directory with the Maven executable (for snapshots) +if [ -z "$actualDistributionDir" ]; then + # enable globbing to iterate over items + set +f + for dir in "$TMP_DOWNLOAD_DIR"/*; do + if [ -d "$dir" ]; then + if [ -f "$dir/bin/$MVN_CMD" ]; then + actualDistributionDir="$(basename "$dir")" + break + fi + fi + done + set -f +fi + +if [ -z "$actualDistributionDir" ]; then + verbose "Contents of $TMP_DOWNLOAD_DIR:" + verbose "$(ls -la "$TMP_DOWNLOAD_DIR")" + die "Could not find Maven distribution directory in extracted archive" +fi + +verbose "Found extracted Maven distribution directory: $actualDistributionDir" +printf %s\\n "$distributionUrl" >"$TMP_DOWNLOAD_DIR/$actualDistributionDir/mvnw.url" +mv -- "$TMP_DOWNLOAD_DIR/$actualDistributionDir" "$MAVEN_HOME" || [ -d "$MAVEN_HOME" ] || die "fail to move MAVEN_HOME" + +clean || : +exec_maven "$@" diff --git a/apache-seata/inventory-service/mvnw.cmd b/apache-seata/inventory-service/mvnw.cmd new file mode 100644 index 000000000000..92450f932734 --- /dev/null +++ b/apache-seata/inventory-service/mvnw.cmd @@ -0,0 +1,189 @@ +<# : batch portion +@REM ---------------------------------------------------------------------------- +@REM Licensed to the Apache Software Foundation (ASF) under one +@REM or more contributor license agreements. See the NOTICE file +@REM distributed with this work for additional information +@REM regarding copyright ownership. The ASF licenses this file +@REM to you under the Apache License, Version 2.0 (the +@REM "License"); you may not use this file except in compliance +@REM with the License. You may obtain a copy of the License at +@REM +@REM http://www.apache.org/licenses/LICENSE-2.0 +@REM +@REM Unless required by applicable law or agreed to in writing, +@REM software distributed under the License is distributed on an +@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +@REM KIND, either express or implied. See the License for the +@REM specific language governing permissions and limitations +@REM under the License. +@REM ---------------------------------------------------------------------------- + +@REM ---------------------------------------------------------------------------- +@REM Apache Maven Wrapper startup batch script, version 3.3.4 +@REM +@REM Optional ENV vars +@REM MVNW_REPOURL - repo url base for downloading maven distribution +@REM MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +@REM MVNW_VERBOSE - true: enable verbose log; others: silence the output +@REM ---------------------------------------------------------------------------- + +@IF "%__MVNW_ARG0_NAME__%"=="" (SET __MVNW_ARG0_NAME__=%~nx0) +@SET __MVNW_CMD__= +@SET __MVNW_ERROR__= +@SET __MVNW_PSMODULEP_SAVE=%PSModulePath% +@SET PSModulePath= +@FOR /F "usebackq tokens=1* delims==" %%A IN (`powershell -noprofile "& {$scriptDir='%~dp0'; $script='%__MVNW_ARG0_NAME__%'; icm -ScriptBlock ([Scriptblock]::Create((Get-Content -Raw '%~f0'))) -NoNewScope}"`) DO @( + IF "%%A"=="MVN_CMD" (set __MVNW_CMD__=%%B) ELSE IF "%%B"=="" (echo %%A) ELSE (echo %%A=%%B) +) +@SET PSModulePath=%__MVNW_PSMODULEP_SAVE% +@SET __MVNW_PSMODULEP_SAVE= +@SET __MVNW_ARG0_NAME__= +@SET MVNW_USERNAME= +@SET MVNW_PASSWORD= +@IF NOT "%__MVNW_CMD__%"=="" ("%__MVNW_CMD__%" %*) +@echo Cannot start maven from wrapper >&2 && exit /b 1 +@GOTO :EOF +: end batch / begin powershell #> + +$ErrorActionPreference = "Stop" +if ($env:MVNW_VERBOSE -eq "true") { + $VerbosePreference = "Continue" +} + +# calculate distributionUrl, requires .mvn/wrapper/maven-wrapper.properties +$distributionUrl = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionUrl +if (!$distributionUrl) { + Write-Error "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" +} + +switch -wildcard -casesensitive ( $($distributionUrl -replace '^.*/','') ) { + "maven-mvnd-*" { + $USE_MVND = $true + $distributionUrl = $distributionUrl -replace '-bin\.[^.]*$',"-windows-amd64.zip" + $MVN_CMD = "mvnd.cmd" + break + } + default { + $USE_MVND = $false + $MVN_CMD = $script -replace '^mvnw','mvn' + break + } +} + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +if ($env:MVNW_REPOURL) { + $MVNW_REPO_PATTERN = if ($USE_MVND -eq $False) { "/org/apache/maven/" } else { "/maven/mvnd/" } + $distributionUrl = "$env:MVNW_REPOURL$MVNW_REPO_PATTERN$($distributionUrl -replace "^.*$MVNW_REPO_PATTERN",'')" +} +$distributionUrlName = $distributionUrl -replace '^.*/','' +$distributionUrlNameMain = $distributionUrlName -replace '\.[^.]*$','' -replace '-bin$','' + +$MAVEN_M2_PATH = "$HOME/.m2" +if ($env:MAVEN_USER_HOME) { + $MAVEN_M2_PATH = "$env:MAVEN_USER_HOME" +} + +if (-not (Test-Path -Path $MAVEN_M2_PATH)) { + New-Item -Path $MAVEN_M2_PATH -ItemType Directory | Out-Null +} + +$MAVEN_WRAPPER_DISTS = $null +if ((Get-Item $MAVEN_M2_PATH).Target[0] -eq $null) { + $MAVEN_WRAPPER_DISTS = "$MAVEN_M2_PATH/wrapper/dists" +} else { + $MAVEN_WRAPPER_DISTS = (Get-Item $MAVEN_M2_PATH).Target[0] + "/wrapper/dists" +} + +$MAVEN_HOME_PARENT = "$MAVEN_WRAPPER_DISTS/$distributionUrlNameMain" +$MAVEN_HOME_NAME = ([System.Security.Cryptography.SHA256]::Create().ComputeHash([byte[]][char[]]$distributionUrl) | ForEach-Object {$_.ToString("x2")}) -join '' +$MAVEN_HOME = "$MAVEN_HOME_PARENT/$MAVEN_HOME_NAME" + +if (Test-Path -Path "$MAVEN_HOME" -PathType Container) { + Write-Verbose "found existing MAVEN_HOME at $MAVEN_HOME" + Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" + exit $? +} + +if (! $distributionUrlNameMain -or ($distributionUrlName -eq $distributionUrlNameMain)) { + Write-Error "distributionUrl is not valid, must end with *-bin.zip, but found $distributionUrl" +} + +# prepare tmp dir +$TMP_DOWNLOAD_DIR_HOLDER = New-TemporaryFile +$TMP_DOWNLOAD_DIR = New-Item -Itemtype Directory -Path "$TMP_DOWNLOAD_DIR_HOLDER.dir" +$TMP_DOWNLOAD_DIR_HOLDER.Delete() | Out-Null +trap { + if ($TMP_DOWNLOAD_DIR.Exists) { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } + } +} + +New-Item -Itemtype Directory -Path "$MAVEN_HOME_PARENT" -Force | Out-Null + +# Download and Install Apache Maven +Write-Verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +Write-Verbose "Downloading from: $distributionUrl" +Write-Verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +$webclient = New-Object System.Net.WebClient +if ($env:MVNW_USERNAME -and $env:MVNW_PASSWORD) { + $webclient.Credentials = New-Object System.Net.NetworkCredential($env:MVNW_USERNAME, $env:MVNW_PASSWORD) +} +[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 +$webclient.DownloadFile($distributionUrl, "$TMP_DOWNLOAD_DIR/$distributionUrlName") | Out-Null + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +$distributionSha256Sum = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionSha256Sum +if ($distributionSha256Sum) { + if ($USE_MVND) { + Write-Error "Checksum validation is not supported for maven-mvnd. `nPlease disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." + } + Import-Module $PSHOME\Modules\Microsoft.PowerShell.Utility -Function Get-FileHash + if ((Get-FileHash "$TMP_DOWNLOAD_DIR/$distributionUrlName" -Algorithm SHA256).Hash.ToLower() -ne $distributionSha256Sum) { + Write-Error "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised. If you updated your Maven version, you need to update the specified distributionSha256Sum property." + } +} + +# unzip and move +Expand-Archive "$TMP_DOWNLOAD_DIR/$distributionUrlName" -DestinationPath "$TMP_DOWNLOAD_DIR" | Out-Null + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +$actualDistributionDir = "" + +# First try the expected directory name (for regular distributions) +$expectedPath = Join-Path "$TMP_DOWNLOAD_DIR" "$distributionUrlNameMain" +$expectedMvnPath = Join-Path "$expectedPath" "bin/$MVN_CMD" +if ((Test-Path -Path $expectedPath -PathType Container) -and (Test-Path -Path $expectedMvnPath -PathType Leaf)) { + $actualDistributionDir = $distributionUrlNameMain +} + +# If not found, search for any directory with the Maven executable (for snapshots) +if (!$actualDistributionDir) { + Get-ChildItem -Path "$TMP_DOWNLOAD_DIR" -Directory | ForEach-Object { + $testPath = Join-Path $_.FullName "bin/$MVN_CMD" + if (Test-Path -Path $testPath -PathType Leaf) { + $actualDistributionDir = $_.Name + } + } +} + +if (!$actualDistributionDir) { + Write-Error "Could not find Maven distribution directory in extracted archive" +} + +Write-Verbose "Found extracted Maven distribution directory: $actualDistributionDir" +Rename-Item -Path "$TMP_DOWNLOAD_DIR/$actualDistributionDir" -NewName $MAVEN_HOME_NAME | Out-Null +try { + Move-Item -Path "$TMP_DOWNLOAD_DIR/$MAVEN_HOME_NAME" -Destination $MAVEN_HOME_PARENT | Out-Null +} catch { + if (! (Test-Path -Path "$MAVEN_HOME" -PathType Container)) { + Write-Error "fail to move MAVEN_HOME" + } +} finally { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } +} + +Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" diff --git a/apache-seata/inventory-service/pom.xml b/apache-seata/inventory-service/pom.xml new file mode 100644 index 000000000000..796d083d2967 --- /dev/null +++ b/apache-seata/inventory-service/pom.xml @@ -0,0 +1,59 @@ + + + 4.0.0 + com.baeldung + apache-seata-inventory-service + 1.0.0-SNAPSHOT + apache-seata-inventory-service + Apache Seata - Inventory Service + + + org.springframework.boot + spring-boot-starter-parent + 3.5.11 + + + + + + org.springframework.boot + spring-boot-starter-jdbc + + + org.springframework.boot + spring-boot-starter-web + + + org.apache.seata + seata-spring-boot-starter + ${seata.version} + + + org.postgresql + postgresql + runtime + + + org.springframework.boot + spring-boot-starter-test + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + + + 17 + 2.6.0 + + + diff --git a/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/ApacheSeataInventoryApplication.java b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/ApacheSeataInventoryApplication.java new file mode 100644 index 000000000000..cf108cb112fb --- /dev/null +++ b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/ApacheSeataInventoryApplication.java @@ -0,0 +1,13 @@ +package com.baeldung.inventory; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +@SpringBootApplication +public class ApacheSeataInventoryApplication { + + public static void main(String[] args) { + SpringApplication.run(ApacheSeataInventoryApplication.class, args); + } + +} diff --git a/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/Controller.java b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/Controller.java new file mode 100644 index 000000000000..fa8bb3b2f097 --- /dev/null +++ b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/Controller.java @@ -0,0 +1,23 @@ +package com.baeldung.inventory; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.transaction.annotation.Transactional; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RestController; + +@RestController +public class Controller { + @Autowired + private Repository repository; + + @PostMapping("/inventory/{mode}") + @Transactional + public void handle(@PathVariable("mode") String mode) { + repository.updateDatabase(); + + if ("inventory".equals(mode)) { + throw new RuntimeException("Inventory Service failed"); + } + } +} diff --git a/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/Repository.java b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/Repository.java new file mode 100644 index 000000000000..44359f2092da --- /dev/null +++ b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/Repository.java @@ -0,0 +1,34 @@ +package com.baeldung.inventory; + +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Map; +import java.util.UUID; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; +import org.springframework.stereotype.Service; + +@Service +public class Repository { + private static final Logger LOG = LoggerFactory.getLogger(Repository.class); + + @Autowired + private NamedParameterJdbcTemplate jdbcTemplate; + + public void updateDatabase() { + var params = Map.of( + "id", UUID.randomUUID().toString(), + "created", Instant.now().atOffset(ZoneOffset.UTC) + ); + + LOG.info("Updating database with {}", params); + + int result = jdbcTemplate.update("INSERT INTO inventory_table(id, created) VALUES (:id, :created)", + params); + + LOG.info("Updating database with result {}", result); + } +} diff --git a/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/SeataXidFilter.java b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/SeataXidFilter.java new file mode 100644 index 000000000000..0da4a2adb78f --- /dev/null +++ b/apache-seata/inventory-service/src/main/java/com/baeldung/inventory/SeataXidFilter.java @@ -0,0 +1,50 @@ +package com.baeldung.inventory; + +import java.io.IOException; + +import org.apache.seata.core.context.RootContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.core.Ordered; +import org.springframework.core.annotation.Order; +import org.springframework.stereotype.Component; +import org.springframework.util.StringUtils; + +import jakarta.servlet.Filter; +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.ServletRequest; +import jakarta.servlet.ServletResponse; +import jakarta.servlet.http.HttpServletRequest; + +@Component +@Order(Ordered.HIGHEST_PRECEDENCE) +public class SeataXidFilter implements Filter { + private static final Logger LOG = LoggerFactory.getLogger(SeataXidFilter.class); + + @Override + public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) + throws IOException, ServletException { + + HttpServletRequest httpRequest = (HttpServletRequest) req; + String xid = httpRequest.getHeader(RootContext.KEY_XID); + + boolean bound = false; + if (StringUtils.hasText(xid) && !xid.equals(RootContext.getXID())) { + LOG.info("Receiving Seata XID: {}", xid); + + RootContext.bind(xid); + bound = true; + } + + try { + chain.doFilter(req, res); + } finally { + // Always unbind — leaking an XID into the next request on this thread + // is a subtle bug that causes phantom branch enrollments. + if (bound) { + RootContext.unbind(); + } + } + } +} \ No newline at end of file diff --git a/apache-seata/inventory-service/src/main/resources/application.properties b/apache-seata/inventory-service/src/main/resources/application.properties new file mode 100644 index 000000000000..5f1e0111d9f7 --- /dev/null +++ b/apache-seata/inventory-service/src/main/resources/application.properties @@ -0,0 +1,25 @@ +spring.application.name=apache-seata-c + +spring.datasource.generate-unique-name=false +spring.datasource.url=jdbc:postgresql://postgres:5432/seata +spring.datasource.username=seata +spring.datasource.password=seata +spring.datasource.driver-class-name=org.postgresql.Driver + +server.port=8081 + +seata.enabled=true +seata.application-id=${spring.application.name} +seata.tx-service-group=my_tx_group + +seata.registry.type=file +seata.registry.file.name=seata.conf + +seata.config.type=file +seata.config.file.name=seata.conf + +seata.service.vgroup-mapping.my_tx_group=default +seata.service.grouplist.default=seata-server:8091 + +seata.data-source-proxy-mode=AT +seata.enable-auto-data-source-proxy=true diff --git a/apache-seata/inventory-service/src/main/resources/seata.conf b/apache-seata/inventory-service/src/main/resources/seata.conf new file mode 100644 index 000000000000..0219477600d9 --- /dev/null +++ b/apache-seata/inventory-service/src/main/resources/seata.conf @@ -0,0 +1,60 @@ +transport { + type = "TCP" + server = "NIO" + heartbeat = true + thread-factory { + boss-thread-prefix = "NettyBoss" + worker-thread-prefix = "NettyServerNIOWorker" + server-executor-thread-size = 100 + share-boss-worker = false + client-selector-thread-size = 1 + client-selector-thread-prefix = "NettyClientSelector" + client-worker-thread-prefix = "NettyClientWorkerThread" + } + shutdown { + wait = 3 + } + serialization = "seata" + compressor = "none" +} + +service { + vgroupMapping.my_tx_group = "default" + default.grouplist = "seata-server:8091" + enableDegrade = false + disableGlobalTransaction = false +} + +client { + rm { + asyncCommitBufferLimit = 10000 + lock { + retryInterval = 10 + retryTimes = 30 + retryPolicyBranchRollbackOnConflict = true + } + reportRetryCount = 5 + tableMetaCheckEnable = false + reportSuccessEnable = false + sagaBranchRegisterEnable = false + } + tm { + commitRetryCount = 5 + rollbackRetryCount = 5 + defaultGlobalTransactionTimeout = 60000 + degradeCheck = false + } + undo { + dataValidation = true + logSerialization = "jackson" + logTable = "undo_log" + compress { + enable = true + type = "zip" + threshold = "64k" + } + } + log { + exceptionRate = 100 + } +} \ No newline at end of file diff --git a/apache-seata/order-service/.dockerignore b/apache-seata/order-service/.dockerignore new file mode 100644 index 000000000000..eb5a316cbd19 --- /dev/null +++ b/apache-seata/order-service/.dockerignore @@ -0,0 +1 @@ +target diff --git a/apache-seata/order-service/.gitattributes b/apache-seata/order-service/.gitattributes new file mode 100644 index 000000000000..3b41682ac579 --- /dev/null +++ b/apache-seata/order-service/.gitattributes @@ -0,0 +1,2 @@ +/mvnw text eol=lf +*.cmd text eol=crlf diff --git a/apache-seata/order-service/.gitignore b/apache-seata/order-service/.gitignore new file mode 100644 index 000000000000..667aaef0c891 --- /dev/null +++ b/apache-seata/order-service/.gitignore @@ -0,0 +1,33 @@ +HELP.md +target/ +.mvn/wrapper/maven-wrapper.jar +!**/src/main/**/target/ +!**/src/test/**/target/ + +### STS ### +.apt_generated +.classpath +.factorypath +.project +.settings +.springBeans +.sts4-cache + +### IntelliJ IDEA ### +.idea +*.iws +*.iml +*.ipr + +### NetBeans ### +/nbproject/private/ +/nbbuild/ +/dist/ +/nbdist/ +/.nb-gradle/ +build/ +!**/src/main/**/build/ +!**/src/test/**/build/ + +### VS Code ### +.vscode/ diff --git a/apache-seata/order-service/.mvn/wrapper/maven-wrapper.properties b/apache-seata/order-service/.mvn/wrapper/maven-wrapper.properties new file mode 100644 index 000000000000..8dea6c227c08 --- /dev/null +++ b/apache-seata/order-service/.mvn/wrapper/maven-wrapper.properties @@ -0,0 +1,3 @@ +wrapperVersion=3.3.4 +distributionType=only-script +distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.12/apache-maven-3.9.12-bin.zip diff --git a/apache-seata/order-service/Dockerfile b/apache-seata/order-service/Dockerfile new file mode 100644 index 000000000000..258559ccad4d --- /dev/null +++ b/apache-seata/order-service/Dockerfile @@ -0,0 +1,18 @@ +FROM maven:3.9.13-eclipse-temurin-17-alpine AS build + +WORKDIR /app + +COPY pom.xml . +RUN mvn compile + +COPY . . +RUN mvn clean package -DskipTests + + + +FROM eclipse-temurin:17 + +WORKDIR /app +COPY --from=build /app/target/*.jar app.jar + +ENTRYPOINT ["java", "-jar", "app.jar"] diff --git a/apache-seata/order-service/mvnw b/apache-seata/order-service/mvnw new file mode 100755 index 000000000000..bd8896bf2217 --- /dev/null +++ b/apache-seata/order-service/mvnw @@ -0,0 +1,295 @@ +#!/bin/sh +# ---------------------------------------------------------------------------- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# ---------------------------------------------------------------------------- + +# ---------------------------------------------------------------------------- +# Apache Maven Wrapper startup batch script, version 3.3.4 +# +# Optional ENV vars +# ----------------- +# JAVA_HOME - location of a JDK home dir, required when download maven via java source +# MVNW_REPOURL - repo url base for downloading maven distribution +# MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +# MVNW_VERBOSE - true: enable verbose log; debug: trace the mvnw script; others: silence the output +# ---------------------------------------------------------------------------- + +set -euf +[ "${MVNW_VERBOSE-}" != debug ] || set -x + +# OS specific support. +native_path() { printf %s\\n "$1"; } +case "$(uname)" in +CYGWIN* | MINGW*) + [ -z "${JAVA_HOME-}" ] || JAVA_HOME="$(cygpath --unix "$JAVA_HOME")" + native_path() { cygpath --path --windows "$1"; } + ;; +esac + +# set JAVACMD and JAVACCMD +set_java_home() { + # For Cygwin and MinGW, ensure paths are in Unix format before anything is touched + if [ -n "${JAVA_HOME-}" ]; then + if [ -x "$JAVA_HOME/jre/sh/java" ]; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + JAVACCMD="$JAVA_HOME/jre/sh/javac" + else + JAVACMD="$JAVA_HOME/bin/java" + JAVACCMD="$JAVA_HOME/bin/javac" + + if [ ! -x "$JAVACMD" ] || [ ! -x "$JAVACCMD" ]; then + echo "The JAVA_HOME environment variable is not defined correctly, so mvnw cannot run." >&2 + echo "JAVA_HOME is set to \"$JAVA_HOME\", but \"\$JAVA_HOME/bin/java\" or \"\$JAVA_HOME/bin/javac\" does not exist." >&2 + return 1 + fi + fi + else + JAVACMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v java + )" || : + JAVACCMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v javac + )" || : + + if [ ! -x "${JAVACMD-}" ] || [ ! -x "${JAVACCMD-}" ]; then + echo "The java/javac command does not exist in PATH nor is JAVA_HOME set, so mvnw cannot run." >&2 + return 1 + fi + fi +} + +# hash string like Java String::hashCode +hash_string() { + str="${1:-}" h=0 + while [ -n "$str" ]; do + char="${str%"${str#?}"}" + h=$(((h * 31 + $(LC_CTYPE=C printf %d "'$char")) % 4294967296)) + str="${str#?}" + done + printf %x\\n $h +} + +verbose() { :; } +[ "${MVNW_VERBOSE-}" != true ] || verbose() { printf %s\\n "${1-}"; } + +die() { + printf %s\\n "$1" >&2 + exit 1 +} + +trim() { + # MWRAPPER-139: + # Trims trailing and leading whitespace, carriage returns, tabs, and linefeeds. + # Needed for removing poorly interpreted newline sequences when running in more + # exotic environments such as mingw bash on Windows. + printf "%s" "${1}" | tr -d '[:space:]' +} + +scriptDir="$(dirname "$0")" +scriptName="$(basename "$0")" + +# parse distributionUrl and optional distributionSha256Sum, requires .mvn/wrapper/maven-wrapper.properties +while IFS="=" read -r key value; do + case "${key-}" in + distributionUrl) distributionUrl=$(trim "${value-}") ;; + distributionSha256Sum) distributionSha256Sum=$(trim "${value-}") ;; + esac +done <"$scriptDir/.mvn/wrapper/maven-wrapper.properties" +[ -n "${distributionUrl-}" ] || die "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" + +case "${distributionUrl##*/}" in +maven-mvnd-*bin.*) + MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ + case "${PROCESSOR_ARCHITECTURE-}${PROCESSOR_ARCHITEW6432-}:$(uname -a)" in + *AMD64:CYGWIN* | *AMD64:MINGW*) distributionPlatform=windows-amd64 ;; + :Darwin*x86_64) distributionPlatform=darwin-amd64 ;; + :Darwin*arm64) distributionPlatform=darwin-aarch64 ;; + :Linux*x86_64*) distributionPlatform=linux-amd64 ;; + *) + echo "Cannot detect native platform for mvnd on $(uname)-$(uname -m), use pure java version" >&2 + distributionPlatform=linux-amd64 + ;; + esac + distributionUrl="${distributionUrl%-bin.*}-$distributionPlatform.zip" + ;; +maven-mvnd-*) MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ ;; +*) MVN_CMD="mvn${scriptName#mvnw}" _MVNW_REPO_PATTERN=/org/apache/maven/ ;; +esac + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +[ -z "${MVNW_REPOURL-}" ] || distributionUrl="$MVNW_REPOURL$_MVNW_REPO_PATTERN${distributionUrl#*"$_MVNW_REPO_PATTERN"}" +distributionUrlName="${distributionUrl##*/}" +distributionUrlNameMain="${distributionUrlName%.*}" +distributionUrlNameMain="${distributionUrlNameMain%-bin}" +MAVEN_USER_HOME="${MAVEN_USER_HOME:-${HOME}/.m2}" +MAVEN_HOME="${MAVEN_USER_HOME}/wrapper/dists/${distributionUrlNameMain-}/$(hash_string "$distributionUrl")" + +exec_maven() { + unset MVNW_VERBOSE MVNW_USERNAME MVNW_PASSWORD MVNW_REPOURL || : + exec "$MAVEN_HOME/bin/$MVN_CMD" "$@" || die "cannot exec $MAVEN_HOME/bin/$MVN_CMD" +} + +if [ -d "$MAVEN_HOME" ]; then + verbose "found existing MAVEN_HOME at $MAVEN_HOME" + exec_maven "$@" +fi + +case "${distributionUrl-}" in +*?-bin.zip | *?maven-mvnd-?*-?*.zip) ;; +*) die "distributionUrl is not valid, must match *-bin.zip or maven-mvnd-*.zip, but found '${distributionUrl-}'" ;; +esac + +# prepare tmp dir +if TMP_DOWNLOAD_DIR="$(mktemp -d)" && [ -d "$TMP_DOWNLOAD_DIR" ]; then + clean() { rm -rf -- "$TMP_DOWNLOAD_DIR"; } + trap clean HUP INT TERM EXIT +else + die "cannot create temp dir" +fi + +mkdir -p -- "${MAVEN_HOME%/*}" + +# Download and Install Apache Maven +verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +verbose "Downloading from: $distributionUrl" +verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +# select .zip or .tar.gz +if ! command -v unzip >/dev/null; then + distributionUrl="${distributionUrl%.zip}.tar.gz" + distributionUrlName="${distributionUrl##*/}" +fi + +# verbose opt +__MVNW_QUIET_WGET=--quiet __MVNW_QUIET_CURL=--silent __MVNW_QUIET_UNZIP=-q __MVNW_QUIET_TAR='' +[ "${MVNW_VERBOSE-}" != true ] || __MVNW_QUIET_WGET='' __MVNW_QUIET_CURL='' __MVNW_QUIET_UNZIP='' __MVNW_QUIET_TAR=v + +# normalize http auth +case "${MVNW_PASSWORD:+has-password}" in +'') MVNW_USERNAME='' MVNW_PASSWORD='' ;; +has-password) [ -n "${MVNW_USERNAME-}" ] || MVNW_USERNAME='' MVNW_PASSWORD='' ;; +esac + +if [ -z "${MVNW_USERNAME-}" ] && command -v wget >/dev/null; then + verbose "Found wget ... using wget" + wget ${__MVNW_QUIET_WGET:+"$__MVNW_QUIET_WGET"} "$distributionUrl" -O "$TMP_DOWNLOAD_DIR/$distributionUrlName" || die "wget: Failed to fetch $distributionUrl" +elif [ -z "${MVNW_USERNAME-}" ] && command -v curl >/dev/null; then + verbose "Found curl ... using curl" + curl ${__MVNW_QUIET_CURL:+"$__MVNW_QUIET_CURL"} -f -L -o "$TMP_DOWNLOAD_DIR/$distributionUrlName" "$distributionUrl" || die "curl: Failed to fetch $distributionUrl" +elif set_java_home; then + verbose "Falling back to use Java to download" + javaSource="$TMP_DOWNLOAD_DIR/Downloader.java" + targetZip="$TMP_DOWNLOAD_DIR/$distributionUrlName" + cat >"$javaSource" <<-END + public class Downloader extends java.net.Authenticator + { + protected java.net.PasswordAuthentication getPasswordAuthentication() + { + return new java.net.PasswordAuthentication( System.getenv( "MVNW_USERNAME" ), System.getenv( "MVNW_PASSWORD" ).toCharArray() ); + } + public static void main( String[] args ) throws Exception + { + setDefault( new Downloader() ); + java.nio.file.Files.copy( java.net.URI.create( args[0] ).toURL().openStream(), java.nio.file.Paths.get( args[1] ).toAbsolutePath().normalize() ); + } + } + END + # For Cygwin/MinGW, switch paths to Windows format before running javac and java + verbose " - Compiling Downloader.java ..." + "$(native_path "$JAVACCMD")" "$(native_path "$javaSource")" || die "Failed to compile Downloader.java" + verbose " - Running Downloader.java ..." + "$(native_path "$JAVACMD")" -cp "$(native_path "$TMP_DOWNLOAD_DIR")" Downloader "$distributionUrl" "$(native_path "$targetZip")" +fi + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +if [ -n "${distributionSha256Sum-}" ]; then + distributionSha256Result=false + if [ "$MVN_CMD" = mvnd.sh ]; then + echo "Checksum validation is not supported for maven-mvnd." >&2 + echo "Please disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + elif command -v sha256sum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | sha256sum -c - >/dev/null 2>&1; then + distributionSha256Result=true + fi + elif command -v shasum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | shasum -a 256 -c >/dev/null 2>&1; then + distributionSha256Result=true + fi + else + echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available." >&2 + echo "Please install either command, or disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + fi + if [ $distributionSha256Result = false ]; then + echo "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised." >&2 + echo "If you updated your Maven version, you need to update the specified distributionSha256Sum property." >&2 + exit 1 + fi +fi + +# unzip and move +if command -v unzip >/dev/null; then + unzip ${__MVNW_QUIET_UNZIP:+"$__MVNW_QUIET_UNZIP"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -d "$TMP_DOWNLOAD_DIR" || die "failed to unzip" +else + tar xzf${__MVNW_QUIET_TAR:+"$__MVNW_QUIET_TAR"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -C "$TMP_DOWNLOAD_DIR" || die "failed to untar" +fi + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +actualDistributionDir="" + +# First try the expected directory name (for regular distributions) +if [ -d "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" ]; then + if [ -f "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain/bin/$MVN_CMD" ]; then + actualDistributionDir="$distributionUrlNameMain" + fi +fi + +# If not found, search for any directory with the Maven executable (for snapshots) +if [ -z "$actualDistributionDir" ]; then + # enable globbing to iterate over items + set +f + for dir in "$TMP_DOWNLOAD_DIR"/*; do + if [ -d "$dir" ]; then + if [ -f "$dir/bin/$MVN_CMD" ]; then + actualDistributionDir="$(basename "$dir")" + break + fi + fi + done + set -f +fi + +if [ -z "$actualDistributionDir" ]; then + verbose "Contents of $TMP_DOWNLOAD_DIR:" + verbose "$(ls -la "$TMP_DOWNLOAD_DIR")" + die "Could not find Maven distribution directory in extracted archive" +fi + +verbose "Found extracted Maven distribution directory: $actualDistributionDir" +printf %s\\n "$distributionUrl" >"$TMP_DOWNLOAD_DIR/$actualDistributionDir/mvnw.url" +mv -- "$TMP_DOWNLOAD_DIR/$actualDistributionDir" "$MAVEN_HOME" || [ -d "$MAVEN_HOME" ] || die "fail to move MAVEN_HOME" + +clean || : +exec_maven "$@" diff --git a/apache-seata/order-service/mvnw.cmd b/apache-seata/order-service/mvnw.cmd new file mode 100644 index 000000000000..92450f932734 --- /dev/null +++ b/apache-seata/order-service/mvnw.cmd @@ -0,0 +1,189 @@ +<# : batch portion +@REM ---------------------------------------------------------------------------- +@REM Licensed to the Apache Software Foundation (ASF) under one +@REM or more contributor license agreements. See the NOTICE file +@REM distributed with this work for additional information +@REM regarding copyright ownership. The ASF licenses this file +@REM to you under the Apache License, Version 2.0 (the +@REM "License"); you may not use this file except in compliance +@REM with the License. You may obtain a copy of the License at +@REM +@REM http://www.apache.org/licenses/LICENSE-2.0 +@REM +@REM Unless required by applicable law or agreed to in writing, +@REM software distributed under the License is distributed on an +@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +@REM KIND, either express or implied. See the License for the +@REM specific language governing permissions and limitations +@REM under the License. +@REM ---------------------------------------------------------------------------- + +@REM ---------------------------------------------------------------------------- +@REM Apache Maven Wrapper startup batch script, version 3.3.4 +@REM +@REM Optional ENV vars +@REM MVNW_REPOURL - repo url base for downloading maven distribution +@REM MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +@REM MVNW_VERBOSE - true: enable verbose log; others: silence the output +@REM ---------------------------------------------------------------------------- + +@IF "%__MVNW_ARG0_NAME__%"=="" (SET __MVNW_ARG0_NAME__=%~nx0) +@SET __MVNW_CMD__= +@SET __MVNW_ERROR__= +@SET __MVNW_PSMODULEP_SAVE=%PSModulePath% +@SET PSModulePath= +@FOR /F "usebackq tokens=1* delims==" %%A IN (`powershell -noprofile "& {$scriptDir='%~dp0'; $script='%__MVNW_ARG0_NAME__%'; icm -ScriptBlock ([Scriptblock]::Create((Get-Content -Raw '%~f0'))) -NoNewScope}"`) DO @( + IF "%%A"=="MVN_CMD" (set __MVNW_CMD__=%%B) ELSE IF "%%B"=="" (echo %%A) ELSE (echo %%A=%%B) +) +@SET PSModulePath=%__MVNW_PSMODULEP_SAVE% +@SET __MVNW_PSMODULEP_SAVE= +@SET __MVNW_ARG0_NAME__= +@SET MVNW_USERNAME= +@SET MVNW_PASSWORD= +@IF NOT "%__MVNW_CMD__%"=="" ("%__MVNW_CMD__%" %*) +@echo Cannot start maven from wrapper >&2 && exit /b 1 +@GOTO :EOF +: end batch / begin powershell #> + +$ErrorActionPreference = "Stop" +if ($env:MVNW_VERBOSE -eq "true") { + $VerbosePreference = "Continue" +} + +# calculate distributionUrl, requires .mvn/wrapper/maven-wrapper.properties +$distributionUrl = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionUrl +if (!$distributionUrl) { + Write-Error "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" +} + +switch -wildcard -casesensitive ( $($distributionUrl -replace '^.*/','') ) { + "maven-mvnd-*" { + $USE_MVND = $true + $distributionUrl = $distributionUrl -replace '-bin\.[^.]*$',"-windows-amd64.zip" + $MVN_CMD = "mvnd.cmd" + break + } + default { + $USE_MVND = $false + $MVN_CMD = $script -replace '^mvnw','mvn' + break + } +} + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +if ($env:MVNW_REPOURL) { + $MVNW_REPO_PATTERN = if ($USE_MVND -eq $False) { "/org/apache/maven/" } else { "/maven/mvnd/" } + $distributionUrl = "$env:MVNW_REPOURL$MVNW_REPO_PATTERN$($distributionUrl -replace "^.*$MVNW_REPO_PATTERN",'')" +} +$distributionUrlName = $distributionUrl -replace '^.*/','' +$distributionUrlNameMain = $distributionUrlName -replace '\.[^.]*$','' -replace '-bin$','' + +$MAVEN_M2_PATH = "$HOME/.m2" +if ($env:MAVEN_USER_HOME) { + $MAVEN_M2_PATH = "$env:MAVEN_USER_HOME" +} + +if (-not (Test-Path -Path $MAVEN_M2_PATH)) { + New-Item -Path $MAVEN_M2_PATH -ItemType Directory | Out-Null +} + +$MAVEN_WRAPPER_DISTS = $null +if ((Get-Item $MAVEN_M2_PATH).Target[0] -eq $null) { + $MAVEN_WRAPPER_DISTS = "$MAVEN_M2_PATH/wrapper/dists" +} else { + $MAVEN_WRAPPER_DISTS = (Get-Item $MAVEN_M2_PATH).Target[0] + "/wrapper/dists" +} + +$MAVEN_HOME_PARENT = "$MAVEN_WRAPPER_DISTS/$distributionUrlNameMain" +$MAVEN_HOME_NAME = ([System.Security.Cryptography.SHA256]::Create().ComputeHash([byte[]][char[]]$distributionUrl) | ForEach-Object {$_.ToString("x2")}) -join '' +$MAVEN_HOME = "$MAVEN_HOME_PARENT/$MAVEN_HOME_NAME" + +if (Test-Path -Path "$MAVEN_HOME" -PathType Container) { + Write-Verbose "found existing MAVEN_HOME at $MAVEN_HOME" + Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" + exit $? +} + +if (! $distributionUrlNameMain -or ($distributionUrlName -eq $distributionUrlNameMain)) { + Write-Error "distributionUrl is not valid, must end with *-bin.zip, but found $distributionUrl" +} + +# prepare tmp dir +$TMP_DOWNLOAD_DIR_HOLDER = New-TemporaryFile +$TMP_DOWNLOAD_DIR = New-Item -Itemtype Directory -Path "$TMP_DOWNLOAD_DIR_HOLDER.dir" +$TMP_DOWNLOAD_DIR_HOLDER.Delete() | Out-Null +trap { + if ($TMP_DOWNLOAD_DIR.Exists) { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } + } +} + +New-Item -Itemtype Directory -Path "$MAVEN_HOME_PARENT" -Force | Out-Null + +# Download and Install Apache Maven +Write-Verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +Write-Verbose "Downloading from: $distributionUrl" +Write-Verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +$webclient = New-Object System.Net.WebClient +if ($env:MVNW_USERNAME -and $env:MVNW_PASSWORD) { + $webclient.Credentials = New-Object System.Net.NetworkCredential($env:MVNW_USERNAME, $env:MVNW_PASSWORD) +} +[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 +$webclient.DownloadFile($distributionUrl, "$TMP_DOWNLOAD_DIR/$distributionUrlName") | Out-Null + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +$distributionSha256Sum = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionSha256Sum +if ($distributionSha256Sum) { + if ($USE_MVND) { + Write-Error "Checksum validation is not supported for maven-mvnd. `nPlease disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." + } + Import-Module $PSHOME\Modules\Microsoft.PowerShell.Utility -Function Get-FileHash + if ((Get-FileHash "$TMP_DOWNLOAD_DIR/$distributionUrlName" -Algorithm SHA256).Hash.ToLower() -ne $distributionSha256Sum) { + Write-Error "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised. If you updated your Maven version, you need to update the specified distributionSha256Sum property." + } +} + +# unzip and move +Expand-Archive "$TMP_DOWNLOAD_DIR/$distributionUrlName" -DestinationPath "$TMP_DOWNLOAD_DIR" | Out-Null + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +$actualDistributionDir = "" + +# First try the expected directory name (for regular distributions) +$expectedPath = Join-Path "$TMP_DOWNLOAD_DIR" "$distributionUrlNameMain" +$expectedMvnPath = Join-Path "$expectedPath" "bin/$MVN_CMD" +if ((Test-Path -Path $expectedPath -PathType Container) -and (Test-Path -Path $expectedMvnPath -PathType Leaf)) { + $actualDistributionDir = $distributionUrlNameMain +} + +# If not found, search for any directory with the Maven executable (for snapshots) +if (!$actualDistributionDir) { + Get-ChildItem -Path "$TMP_DOWNLOAD_DIR" -Directory | ForEach-Object { + $testPath = Join-Path $_.FullName "bin/$MVN_CMD" + if (Test-Path -Path $testPath -PathType Leaf) { + $actualDistributionDir = $_.Name + } + } +} + +if (!$actualDistributionDir) { + Write-Error "Could not find Maven distribution directory in extracted archive" +} + +Write-Verbose "Found extracted Maven distribution directory: $actualDistributionDir" +Rename-Item -Path "$TMP_DOWNLOAD_DIR/$actualDistributionDir" -NewName $MAVEN_HOME_NAME | Out-Null +try { + Move-Item -Path "$TMP_DOWNLOAD_DIR/$MAVEN_HOME_NAME" -Destination $MAVEN_HOME_PARENT | Out-Null +} catch { + if (! (Test-Path -Path "$MAVEN_HOME" -PathType Container)) { + Write-Error "fail to move MAVEN_HOME" + } +} finally { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } +} + +Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" diff --git a/apache-seata/order-service/pom.xml b/apache-seata/order-service/pom.xml new file mode 100644 index 000000000000..f29b6928e089 --- /dev/null +++ b/apache-seata/order-service/pom.xml @@ -0,0 +1,59 @@ + + + 4.0.0 + com.baeldung + apache-seata-order-service + 1.0.0-SNAPSHOT + apache-seata-order-service + Apache Seata - Order Service + + + org.springframework.boot + spring-boot-starter-parent + 3.5.11 + + + + + + org.springframework.boot + spring-boot-starter-jdbc + + + org.springframework.boot + spring-boot-starter-web + + + org.apache.seata + seata-spring-boot-starter + ${seata.version} + + + org.postgresql + postgresql + runtime + + + org.springframework.boot + spring-boot-starter-test + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + + + 17 + 2.6.0 + + + diff --git a/apache-seata/order-service/src/main/java/com/baeldung/order/ApacheSeataOrderApplication.java b/apache-seata/order-service/src/main/java/com/baeldung/order/ApacheSeataOrderApplication.java new file mode 100644 index 000000000000..1fb370e89f8f --- /dev/null +++ b/apache-seata/order-service/src/main/java/com/baeldung/order/ApacheSeataOrderApplication.java @@ -0,0 +1,13 @@ +package com.baeldung.order; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +@SpringBootApplication +public class ApacheSeataOrderApplication { + + public static void main(String[] args) { + SpringApplication.run(ApacheSeataOrderApplication.class, args); + } + +} diff --git a/apache-seata/order-service/src/main/java/com/baeldung/order/Controller.java b/apache-seata/order-service/src/main/java/com/baeldung/order/Controller.java new file mode 100644 index 000000000000..3d43378bf0d1 --- /dev/null +++ b/apache-seata/order-service/src/main/java/com/baeldung/order/Controller.java @@ -0,0 +1,23 @@ +package com.baeldung.order; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.transaction.annotation.Transactional; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RestController; + +@RestController +public class Controller { + @Autowired + private Repository repository; + + @PostMapping("/order/{mode}") + @Transactional + public void handle(@PathVariable("mode") String mode) { + repository.updateDatabase(); + + if ("order".equals(mode)) { + throw new RuntimeException("Order Service failed"); + } + } +} diff --git a/apache-seata/order-service/src/main/java/com/baeldung/order/Repository.java b/apache-seata/order-service/src/main/java/com/baeldung/order/Repository.java new file mode 100644 index 000000000000..16721104deb9 --- /dev/null +++ b/apache-seata/order-service/src/main/java/com/baeldung/order/Repository.java @@ -0,0 +1,34 @@ +package com.baeldung.order; + +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Map; +import java.util.UUID; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; +import org.springframework.stereotype.Service; + +@Service +public class Repository { + private static final Logger LOG = LoggerFactory.getLogger(Repository.class); + + @Autowired + private NamedParameterJdbcTemplate jdbcTemplate; + + public void updateDatabase() { + var params = Map.of( + "id", UUID.randomUUID().toString(), + "created", Instant.now().atOffset(ZoneOffset.UTC) + ); + + LOG.info("Updating database with {}", params); + + int result = jdbcTemplate.update("INSERT INTO order_table(id, created) VALUES (:id, :created)", + params); + + LOG.info("Updating database with result {}", result); + } +} diff --git a/apache-seata/order-service/src/main/java/com/baeldung/order/SeataXidFilter.java b/apache-seata/order-service/src/main/java/com/baeldung/order/SeataXidFilter.java new file mode 100644 index 000000000000..75aeacbc52e6 --- /dev/null +++ b/apache-seata/order-service/src/main/java/com/baeldung/order/SeataXidFilter.java @@ -0,0 +1,50 @@ +package com.baeldung.order; + +import java.io.IOException; + +import org.apache.seata.core.context.RootContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.core.Ordered; +import org.springframework.core.annotation.Order; +import org.springframework.stereotype.Component; +import org.springframework.util.StringUtils; + +import jakarta.servlet.Filter; +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.ServletRequest; +import jakarta.servlet.ServletResponse; +import jakarta.servlet.http.HttpServletRequest; + +@Component +@Order(Ordered.HIGHEST_PRECEDENCE) +public class SeataXidFilter implements Filter { + private static final Logger LOG = LoggerFactory.getLogger(SeataXidFilter.class); + + @Override + public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) + throws IOException, ServletException { + + HttpServletRequest httpRequest = (HttpServletRequest) req; + String xid = httpRequest.getHeader(RootContext.KEY_XID); + + boolean bound = false; + if (StringUtils.hasText(xid) && !xid.equals(RootContext.getXID())) { + LOG.info("Receiving Seata XID: {}", xid); + + RootContext.bind(xid); + bound = true; + } + + try { + chain.doFilter(req, res); + } finally { + // Always unbind — leaking an XID into the next request on this thread + // is a subtle bug that causes phantom branch enrollments. + if (bound) { + RootContext.unbind(); + } + } + } +} \ No newline at end of file diff --git a/apache-seata/order-service/src/main/resources/application.properties b/apache-seata/order-service/src/main/resources/application.properties new file mode 100644 index 000000000000..d2cbdfc8ead5 --- /dev/null +++ b/apache-seata/order-service/src/main/resources/application.properties @@ -0,0 +1,25 @@ +spring.application.name=apache-seata-c + +spring.datasource.generate-unique-name=false +spring.datasource.url=jdbc:postgresql://postgres:5432/seata +spring.datasource.username=seata +spring.datasource.password=seata +spring.datasource.driver-class-name=org.postgresql.Driver + +server.port=8082 + +seata.enabled=true +seata.application-id=${spring.application.name} +seata.tx-service-group=my_tx_group + +seata.registry.type=file +seata.registry.file.name=seata.conf + +seata.config.type=file +seata.config.file.name=seata.conf + +seata.service.vgroup-mapping.my_tx_group=default +seata.service.grouplist.default=seata-server:8091 + +seata.data-source-proxy-mode=AT +seata.enable-auto-data-source-proxy=true \ No newline at end of file diff --git a/apache-seata/order-service/src/main/resources/seata.conf b/apache-seata/order-service/src/main/resources/seata.conf new file mode 100644 index 000000000000..0219477600d9 --- /dev/null +++ b/apache-seata/order-service/src/main/resources/seata.conf @@ -0,0 +1,60 @@ +transport { + type = "TCP" + server = "NIO" + heartbeat = true + thread-factory { + boss-thread-prefix = "NettyBoss" + worker-thread-prefix = "NettyServerNIOWorker" + server-executor-thread-size = 100 + share-boss-worker = false + client-selector-thread-size = 1 + client-selector-thread-prefix = "NettyClientSelector" + client-worker-thread-prefix = "NettyClientWorkerThread" + } + shutdown { + wait = 3 + } + serialization = "seata" + compressor = "none" +} + +service { + vgroupMapping.my_tx_group = "default" + default.grouplist = "seata-server:8091" + enableDegrade = false + disableGlobalTransaction = false +} + +client { + rm { + asyncCommitBufferLimit = 10000 + lock { + retryInterval = 10 + retryTimes = 30 + retryPolicyBranchRollbackOnConflict = true + } + reportRetryCount = 5 + tableMetaCheckEnable = false + reportSuccessEnable = false + sagaBranchRegisterEnable = false + } + tm { + commitRetryCount = 5 + rollbackRetryCount = 5 + defaultGlobalTransactionTimeout = 60000 + degradeCheck = false + } + undo { + dataValidation = true + logSerialization = "jackson" + logTable = "undo_log" + compress { + enable = true + type = "zip" + threshold = "64k" + } + } + log { + exceptionRate = 100 + } +} \ No newline at end of file diff --git a/apache-seata/pom.xml b/apache-seata/pom.xml new file mode 100644 index 000000000000..f20cb92dabe4 --- /dev/null +++ b/apache-seata/pom.xml @@ -0,0 +1,19 @@ + + + + 4.0.0 + com.baeldung + parent-apache-seata + 1.0.0-SNAPSHOT + pom + parent-apache-seata + + + shop-service + inventory-service + order-service + billing-service + + diff --git a/apache-seata/requests/shop.http b/apache-seata/requests/shop.http new file mode 100644 index 000000000000..2c1296395195 --- /dev/null +++ b/apache-seata/requests/shop.http @@ -0,0 +1,16 @@ +# Success +POST http://localhost:8080/shop/success HTTP/1.1 + +### +# Fails in Inventory service +POST http://localhost:8080/shop/inventory HTTP/1.1 + +### +# Fails in Order service +POST http://localhost:8080/shop/order HTTP/1.1 + +### +# Fails in Billing service +POST http://localhost:8080/shop/blling HTTP/1.1 + +### diff --git a/apache-seata/shop-service/.dockerignore b/apache-seata/shop-service/.dockerignore new file mode 100644 index 000000000000..eb5a316cbd19 --- /dev/null +++ b/apache-seata/shop-service/.dockerignore @@ -0,0 +1 @@ +target diff --git a/apache-seata/shop-service/.gitattributes b/apache-seata/shop-service/.gitattributes new file mode 100644 index 000000000000..3b41682ac579 --- /dev/null +++ b/apache-seata/shop-service/.gitattributes @@ -0,0 +1,2 @@ +/mvnw text eol=lf +*.cmd text eol=crlf diff --git a/apache-seata/shop-service/.gitignore b/apache-seata/shop-service/.gitignore new file mode 100644 index 000000000000..667aaef0c891 --- /dev/null +++ b/apache-seata/shop-service/.gitignore @@ -0,0 +1,33 @@ +HELP.md +target/ +.mvn/wrapper/maven-wrapper.jar +!**/src/main/**/target/ +!**/src/test/**/target/ + +### STS ### +.apt_generated +.classpath +.factorypath +.project +.settings +.springBeans +.sts4-cache + +### IntelliJ IDEA ### +.idea +*.iws +*.iml +*.ipr + +### NetBeans ### +/nbproject/private/ +/nbbuild/ +/dist/ +/nbdist/ +/.nb-gradle/ +build/ +!**/src/main/**/build/ +!**/src/test/**/build/ + +### VS Code ### +.vscode/ diff --git a/apache-seata/shop-service/.mvn/wrapper/maven-wrapper.properties b/apache-seata/shop-service/.mvn/wrapper/maven-wrapper.properties new file mode 100644 index 000000000000..8dea6c227c08 --- /dev/null +++ b/apache-seata/shop-service/.mvn/wrapper/maven-wrapper.properties @@ -0,0 +1,3 @@ +wrapperVersion=3.3.4 +distributionType=only-script +distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.12/apache-maven-3.9.12-bin.zip diff --git a/apache-seata/shop-service/Dockerfile b/apache-seata/shop-service/Dockerfile new file mode 100644 index 000000000000..258559ccad4d --- /dev/null +++ b/apache-seata/shop-service/Dockerfile @@ -0,0 +1,18 @@ +FROM maven:3.9.13-eclipse-temurin-17-alpine AS build + +WORKDIR /app + +COPY pom.xml . +RUN mvn compile + +COPY . . +RUN mvn clean package -DskipTests + + + +FROM eclipse-temurin:17 + +WORKDIR /app +COPY --from=build /app/target/*.jar app.jar + +ENTRYPOINT ["java", "-jar", "app.jar"] diff --git a/apache-seata/shop-service/mvnw b/apache-seata/shop-service/mvnw new file mode 100755 index 000000000000..bd8896bf2217 --- /dev/null +++ b/apache-seata/shop-service/mvnw @@ -0,0 +1,295 @@ +#!/bin/sh +# ---------------------------------------------------------------------------- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# ---------------------------------------------------------------------------- + +# ---------------------------------------------------------------------------- +# Apache Maven Wrapper startup batch script, version 3.3.4 +# +# Optional ENV vars +# ----------------- +# JAVA_HOME - location of a JDK home dir, required when download maven via java source +# MVNW_REPOURL - repo url base for downloading maven distribution +# MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +# MVNW_VERBOSE - true: enable verbose log; debug: trace the mvnw script; others: silence the output +# ---------------------------------------------------------------------------- + +set -euf +[ "${MVNW_VERBOSE-}" != debug ] || set -x + +# OS specific support. +native_path() { printf %s\\n "$1"; } +case "$(uname)" in +CYGWIN* | MINGW*) + [ -z "${JAVA_HOME-}" ] || JAVA_HOME="$(cygpath --unix "$JAVA_HOME")" + native_path() { cygpath --path --windows "$1"; } + ;; +esac + +# set JAVACMD and JAVACCMD +set_java_home() { + # For Cygwin and MinGW, ensure paths are in Unix format before anything is touched + if [ -n "${JAVA_HOME-}" ]; then + if [ -x "$JAVA_HOME/jre/sh/java" ]; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + JAVACCMD="$JAVA_HOME/jre/sh/javac" + else + JAVACMD="$JAVA_HOME/bin/java" + JAVACCMD="$JAVA_HOME/bin/javac" + + if [ ! -x "$JAVACMD" ] || [ ! -x "$JAVACCMD" ]; then + echo "The JAVA_HOME environment variable is not defined correctly, so mvnw cannot run." >&2 + echo "JAVA_HOME is set to \"$JAVA_HOME\", but \"\$JAVA_HOME/bin/java\" or \"\$JAVA_HOME/bin/javac\" does not exist." >&2 + return 1 + fi + fi + else + JAVACMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v java + )" || : + JAVACCMD="$( + 'set' +e + 'unset' -f command 2>/dev/null + 'command' -v javac + )" || : + + if [ ! -x "${JAVACMD-}" ] || [ ! -x "${JAVACCMD-}" ]; then + echo "The java/javac command does not exist in PATH nor is JAVA_HOME set, so mvnw cannot run." >&2 + return 1 + fi + fi +} + +# hash string like Java String::hashCode +hash_string() { + str="${1:-}" h=0 + while [ -n "$str" ]; do + char="${str%"${str#?}"}" + h=$(((h * 31 + $(LC_CTYPE=C printf %d "'$char")) % 4294967296)) + str="${str#?}" + done + printf %x\\n $h +} + +verbose() { :; } +[ "${MVNW_VERBOSE-}" != true ] || verbose() { printf %s\\n "${1-}"; } + +die() { + printf %s\\n "$1" >&2 + exit 1 +} + +trim() { + # MWRAPPER-139: + # Trims trailing and leading whitespace, carriage returns, tabs, and linefeeds. + # Needed for removing poorly interpreted newline sequences when running in more + # exotic environments such as mingw bash on Windows. + printf "%s" "${1}" | tr -d '[:space:]' +} + +scriptDir="$(dirname "$0")" +scriptName="$(basename "$0")" + +# parse distributionUrl and optional distributionSha256Sum, requires .mvn/wrapper/maven-wrapper.properties +while IFS="=" read -r key value; do + case "${key-}" in + distributionUrl) distributionUrl=$(trim "${value-}") ;; + distributionSha256Sum) distributionSha256Sum=$(trim "${value-}") ;; + esac +done <"$scriptDir/.mvn/wrapper/maven-wrapper.properties" +[ -n "${distributionUrl-}" ] || die "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" + +case "${distributionUrl##*/}" in +maven-mvnd-*bin.*) + MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ + case "${PROCESSOR_ARCHITECTURE-}${PROCESSOR_ARCHITEW6432-}:$(uname -a)" in + *AMD64:CYGWIN* | *AMD64:MINGW*) distributionPlatform=windows-amd64 ;; + :Darwin*x86_64) distributionPlatform=darwin-amd64 ;; + :Darwin*arm64) distributionPlatform=darwin-aarch64 ;; + :Linux*x86_64*) distributionPlatform=linux-amd64 ;; + *) + echo "Cannot detect native platform for mvnd on $(uname)-$(uname -m), use pure java version" >&2 + distributionPlatform=linux-amd64 + ;; + esac + distributionUrl="${distributionUrl%-bin.*}-$distributionPlatform.zip" + ;; +maven-mvnd-*) MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ ;; +*) MVN_CMD="mvn${scriptName#mvnw}" _MVNW_REPO_PATTERN=/org/apache/maven/ ;; +esac + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +[ -z "${MVNW_REPOURL-}" ] || distributionUrl="$MVNW_REPOURL$_MVNW_REPO_PATTERN${distributionUrl#*"$_MVNW_REPO_PATTERN"}" +distributionUrlName="${distributionUrl##*/}" +distributionUrlNameMain="${distributionUrlName%.*}" +distributionUrlNameMain="${distributionUrlNameMain%-bin}" +MAVEN_USER_HOME="${MAVEN_USER_HOME:-${HOME}/.m2}" +MAVEN_HOME="${MAVEN_USER_HOME}/wrapper/dists/${distributionUrlNameMain-}/$(hash_string "$distributionUrl")" + +exec_maven() { + unset MVNW_VERBOSE MVNW_USERNAME MVNW_PASSWORD MVNW_REPOURL || : + exec "$MAVEN_HOME/bin/$MVN_CMD" "$@" || die "cannot exec $MAVEN_HOME/bin/$MVN_CMD" +} + +if [ -d "$MAVEN_HOME" ]; then + verbose "found existing MAVEN_HOME at $MAVEN_HOME" + exec_maven "$@" +fi + +case "${distributionUrl-}" in +*?-bin.zip | *?maven-mvnd-?*-?*.zip) ;; +*) die "distributionUrl is not valid, must match *-bin.zip or maven-mvnd-*.zip, but found '${distributionUrl-}'" ;; +esac + +# prepare tmp dir +if TMP_DOWNLOAD_DIR="$(mktemp -d)" && [ -d "$TMP_DOWNLOAD_DIR" ]; then + clean() { rm -rf -- "$TMP_DOWNLOAD_DIR"; } + trap clean HUP INT TERM EXIT +else + die "cannot create temp dir" +fi + +mkdir -p -- "${MAVEN_HOME%/*}" + +# Download and Install Apache Maven +verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +verbose "Downloading from: $distributionUrl" +verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +# select .zip or .tar.gz +if ! command -v unzip >/dev/null; then + distributionUrl="${distributionUrl%.zip}.tar.gz" + distributionUrlName="${distributionUrl##*/}" +fi + +# verbose opt +__MVNW_QUIET_WGET=--quiet __MVNW_QUIET_CURL=--silent __MVNW_QUIET_UNZIP=-q __MVNW_QUIET_TAR='' +[ "${MVNW_VERBOSE-}" != true ] || __MVNW_QUIET_WGET='' __MVNW_QUIET_CURL='' __MVNW_QUIET_UNZIP='' __MVNW_QUIET_TAR=v + +# normalize http auth +case "${MVNW_PASSWORD:+has-password}" in +'') MVNW_USERNAME='' MVNW_PASSWORD='' ;; +has-password) [ -n "${MVNW_USERNAME-}" ] || MVNW_USERNAME='' MVNW_PASSWORD='' ;; +esac + +if [ -z "${MVNW_USERNAME-}" ] && command -v wget >/dev/null; then + verbose "Found wget ... using wget" + wget ${__MVNW_QUIET_WGET:+"$__MVNW_QUIET_WGET"} "$distributionUrl" -O "$TMP_DOWNLOAD_DIR/$distributionUrlName" || die "wget: Failed to fetch $distributionUrl" +elif [ -z "${MVNW_USERNAME-}" ] && command -v curl >/dev/null; then + verbose "Found curl ... using curl" + curl ${__MVNW_QUIET_CURL:+"$__MVNW_QUIET_CURL"} -f -L -o "$TMP_DOWNLOAD_DIR/$distributionUrlName" "$distributionUrl" || die "curl: Failed to fetch $distributionUrl" +elif set_java_home; then + verbose "Falling back to use Java to download" + javaSource="$TMP_DOWNLOAD_DIR/Downloader.java" + targetZip="$TMP_DOWNLOAD_DIR/$distributionUrlName" + cat >"$javaSource" <<-END + public class Downloader extends java.net.Authenticator + { + protected java.net.PasswordAuthentication getPasswordAuthentication() + { + return new java.net.PasswordAuthentication( System.getenv( "MVNW_USERNAME" ), System.getenv( "MVNW_PASSWORD" ).toCharArray() ); + } + public static void main( String[] args ) throws Exception + { + setDefault( new Downloader() ); + java.nio.file.Files.copy( java.net.URI.create( args[0] ).toURL().openStream(), java.nio.file.Paths.get( args[1] ).toAbsolutePath().normalize() ); + } + } + END + # For Cygwin/MinGW, switch paths to Windows format before running javac and java + verbose " - Compiling Downloader.java ..." + "$(native_path "$JAVACCMD")" "$(native_path "$javaSource")" || die "Failed to compile Downloader.java" + verbose " - Running Downloader.java ..." + "$(native_path "$JAVACMD")" -cp "$(native_path "$TMP_DOWNLOAD_DIR")" Downloader "$distributionUrl" "$(native_path "$targetZip")" +fi + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +if [ -n "${distributionSha256Sum-}" ]; then + distributionSha256Result=false + if [ "$MVN_CMD" = mvnd.sh ]; then + echo "Checksum validation is not supported for maven-mvnd." >&2 + echo "Please disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + elif command -v sha256sum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | sha256sum -c - >/dev/null 2>&1; then + distributionSha256Result=true + fi + elif command -v shasum >/dev/null; then + if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | shasum -a 256 -c >/dev/null 2>&1; then + distributionSha256Result=true + fi + else + echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available." >&2 + echo "Please install either command, or disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 + exit 1 + fi + if [ $distributionSha256Result = false ]; then + echo "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised." >&2 + echo "If you updated your Maven version, you need to update the specified distributionSha256Sum property." >&2 + exit 1 + fi +fi + +# unzip and move +if command -v unzip >/dev/null; then + unzip ${__MVNW_QUIET_UNZIP:+"$__MVNW_QUIET_UNZIP"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -d "$TMP_DOWNLOAD_DIR" || die "failed to unzip" +else + tar xzf${__MVNW_QUIET_TAR:+"$__MVNW_QUIET_TAR"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -C "$TMP_DOWNLOAD_DIR" || die "failed to untar" +fi + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +actualDistributionDir="" + +# First try the expected directory name (for regular distributions) +if [ -d "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" ]; then + if [ -f "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain/bin/$MVN_CMD" ]; then + actualDistributionDir="$distributionUrlNameMain" + fi +fi + +# If not found, search for any directory with the Maven executable (for snapshots) +if [ -z "$actualDistributionDir" ]; then + # enable globbing to iterate over items + set +f + for dir in "$TMP_DOWNLOAD_DIR"/*; do + if [ -d "$dir" ]; then + if [ -f "$dir/bin/$MVN_CMD" ]; then + actualDistributionDir="$(basename "$dir")" + break + fi + fi + done + set -f +fi + +if [ -z "$actualDistributionDir" ]; then + verbose "Contents of $TMP_DOWNLOAD_DIR:" + verbose "$(ls -la "$TMP_DOWNLOAD_DIR")" + die "Could not find Maven distribution directory in extracted archive" +fi + +verbose "Found extracted Maven distribution directory: $actualDistributionDir" +printf %s\\n "$distributionUrl" >"$TMP_DOWNLOAD_DIR/$actualDistributionDir/mvnw.url" +mv -- "$TMP_DOWNLOAD_DIR/$actualDistributionDir" "$MAVEN_HOME" || [ -d "$MAVEN_HOME" ] || die "fail to move MAVEN_HOME" + +clean || : +exec_maven "$@" diff --git a/apache-seata/shop-service/mvnw.cmd b/apache-seata/shop-service/mvnw.cmd new file mode 100644 index 000000000000..92450f932734 --- /dev/null +++ b/apache-seata/shop-service/mvnw.cmd @@ -0,0 +1,189 @@ +<# : batch portion +@REM ---------------------------------------------------------------------------- +@REM Licensed to the Apache Software Foundation (ASF) under one +@REM or more contributor license agreements. See the NOTICE file +@REM distributed with this work for additional information +@REM regarding copyright ownership. The ASF licenses this file +@REM to you under the Apache License, Version 2.0 (the +@REM "License"); you may not use this file except in compliance +@REM with the License. You may obtain a copy of the License at +@REM +@REM http://www.apache.org/licenses/LICENSE-2.0 +@REM +@REM Unless required by applicable law or agreed to in writing, +@REM software distributed under the License is distributed on an +@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +@REM KIND, either express or implied. See the License for the +@REM specific language governing permissions and limitations +@REM under the License. +@REM ---------------------------------------------------------------------------- + +@REM ---------------------------------------------------------------------------- +@REM Apache Maven Wrapper startup batch script, version 3.3.4 +@REM +@REM Optional ENV vars +@REM MVNW_REPOURL - repo url base for downloading maven distribution +@REM MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven +@REM MVNW_VERBOSE - true: enable verbose log; others: silence the output +@REM ---------------------------------------------------------------------------- + +@IF "%__MVNW_ARG0_NAME__%"=="" (SET __MVNW_ARG0_NAME__=%~nx0) +@SET __MVNW_CMD__= +@SET __MVNW_ERROR__= +@SET __MVNW_PSMODULEP_SAVE=%PSModulePath% +@SET PSModulePath= +@FOR /F "usebackq tokens=1* delims==" %%A IN (`powershell -noprofile "& {$scriptDir='%~dp0'; $script='%__MVNW_ARG0_NAME__%'; icm -ScriptBlock ([Scriptblock]::Create((Get-Content -Raw '%~f0'))) -NoNewScope}"`) DO @( + IF "%%A"=="MVN_CMD" (set __MVNW_CMD__=%%B) ELSE IF "%%B"=="" (echo %%A) ELSE (echo %%A=%%B) +) +@SET PSModulePath=%__MVNW_PSMODULEP_SAVE% +@SET __MVNW_PSMODULEP_SAVE= +@SET __MVNW_ARG0_NAME__= +@SET MVNW_USERNAME= +@SET MVNW_PASSWORD= +@IF NOT "%__MVNW_CMD__%"=="" ("%__MVNW_CMD__%" %*) +@echo Cannot start maven from wrapper >&2 && exit /b 1 +@GOTO :EOF +: end batch / begin powershell #> + +$ErrorActionPreference = "Stop" +if ($env:MVNW_VERBOSE -eq "true") { + $VerbosePreference = "Continue" +} + +# calculate distributionUrl, requires .mvn/wrapper/maven-wrapper.properties +$distributionUrl = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionUrl +if (!$distributionUrl) { + Write-Error "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" +} + +switch -wildcard -casesensitive ( $($distributionUrl -replace '^.*/','') ) { + "maven-mvnd-*" { + $USE_MVND = $true + $distributionUrl = $distributionUrl -replace '-bin\.[^.]*$',"-windows-amd64.zip" + $MVN_CMD = "mvnd.cmd" + break + } + default { + $USE_MVND = $false + $MVN_CMD = $script -replace '^mvnw','mvn' + break + } +} + +# apply MVNW_REPOURL and calculate MAVEN_HOME +# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ +if ($env:MVNW_REPOURL) { + $MVNW_REPO_PATTERN = if ($USE_MVND -eq $False) { "/org/apache/maven/" } else { "/maven/mvnd/" } + $distributionUrl = "$env:MVNW_REPOURL$MVNW_REPO_PATTERN$($distributionUrl -replace "^.*$MVNW_REPO_PATTERN",'')" +} +$distributionUrlName = $distributionUrl -replace '^.*/','' +$distributionUrlNameMain = $distributionUrlName -replace '\.[^.]*$','' -replace '-bin$','' + +$MAVEN_M2_PATH = "$HOME/.m2" +if ($env:MAVEN_USER_HOME) { + $MAVEN_M2_PATH = "$env:MAVEN_USER_HOME" +} + +if (-not (Test-Path -Path $MAVEN_M2_PATH)) { + New-Item -Path $MAVEN_M2_PATH -ItemType Directory | Out-Null +} + +$MAVEN_WRAPPER_DISTS = $null +if ((Get-Item $MAVEN_M2_PATH).Target[0] -eq $null) { + $MAVEN_WRAPPER_DISTS = "$MAVEN_M2_PATH/wrapper/dists" +} else { + $MAVEN_WRAPPER_DISTS = (Get-Item $MAVEN_M2_PATH).Target[0] + "/wrapper/dists" +} + +$MAVEN_HOME_PARENT = "$MAVEN_WRAPPER_DISTS/$distributionUrlNameMain" +$MAVEN_HOME_NAME = ([System.Security.Cryptography.SHA256]::Create().ComputeHash([byte[]][char[]]$distributionUrl) | ForEach-Object {$_.ToString("x2")}) -join '' +$MAVEN_HOME = "$MAVEN_HOME_PARENT/$MAVEN_HOME_NAME" + +if (Test-Path -Path "$MAVEN_HOME" -PathType Container) { + Write-Verbose "found existing MAVEN_HOME at $MAVEN_HOME" + Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" + exit $? +} + +if (! $distributionUrlNameMain -or ($distributionUrlName -eq $distributionUrlNameMain)) { + Write-Error "distributionUrl is not valid, must end with *-bin.zip, but found $distributionUrl" +} + +# prepare tmp dir +$TMP_DOWNLOAD_DIR_HOLDER = New-TemporaryFile +$TMP_DOWNLOAD_DIR = New-Item -Itemtype Directory -Path "$TMP_DOWNLOAD_DIR_HOLDER.dir" +$TMP_DOWNLOAD_DIR_HOLDER.Delete() | Out-Null +trap { + if ($TMP_DOWNLOAD_DIR.Exists) { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } + } +} + +New-Item -Itemtype Directory -Path "$MAVEN_HOME_PARENT" -Force | Out-Null + +# Download and Install Apache Maven +Write-Verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." +Write-Verbose "Downloading from: $distributionUrl" +Write-Verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" + +$webclient = New-Object System.Net.WebClient +if ($env:MVNW_USERNAME -and $env:MVNW_PASSWORD) { + $webclient.Credentials = New-Object System.Net.NetworkCredential($env:MVNW_USERNAME, $env:MVNW_PASSWORD) +} +[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 +$webclient.DownloadFile($distributionUrl, "$TMP_DOWNLOAD_DIR/$distributionUrlName") | Out-Null + +# If specified, validate the SHA-256 sum of the Maven distribution zip file +$distributionSha256Sum = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionSha256Sum +if ($distributionSha256Sum) { + if ($USE_MVND) { + Write-Error "Checksum validation is not supported for maven-mvnd. `nPlease disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." + } + Import-Module $PSHOME\Modules\Microsoft.PowerShell.Utility -Function Get-FileHash + if ((Get-FileHash "$TMP_DOWNLOAD_DIR/$distributionUrlName" -Algorithm SHA256).Hash.ToLower() -ne $distributionSha256Sum) { + Write-Error "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised. If you updated your Maven version, you need to update the specified distributionSha256Sum property." + } +} + +# unzip and move +Expand-Archive "$TMP_DOWNLOAD_DIR/$distributionUrlName" -DestinationPath "$TMP_DOWNLOAD_DIR" | Out-Null + +# Find the actual extracted directory name (handles snapshots where filename != directory name) +$actualDistributionDir = "" + +# First try the expected directory name (for regular distributions) +$expectedPath = Join-Path "$TMP_DOWNLOAD_DIR" "$distributionUrlNameMain" +$expectedMvnPath = Join-Path "$expectedPath" "bin/$MVN_CMD" +if ((Test-Path -Path $expectedPath -PathType Container) -and (Test-Path -Path $expectedMvnPath -PathType Leaf)) { + $actualDistributionDir = $distributionUrlNameMain +} + +# If not found, search for any directory with the Maven executable (for snapshots) +if (!$actualDistributionDir) { + Get-ChildItem -Path "$TMP_DOWNLOAD_DIR" -Directory | ForEach-Object { + $testPath = Join-Path $_.FullName "bin/$MVN_CMD" + if (Test-Path -Path $testPath -PathType Leaf) { + $actualDistributionDir = $_.Name + } + } +} + +if (!$actualDistributionDir) { + Write-Error "Could not find Maven distribution directory in extracted archive" +} + +Write-Verbose "Found extracted Maven distribution directory: $actualDistributionDir" +Rename-Item -Path "$TMP_DOWNLOAD_DIR/$actualDistributionDir" -NewName $MAVEN_HOME_NAME | Out-Null +try { + Move-Item -Path "$TMP_DOWNLOAD_DIR/$MAVEN_HOME_NAME" -Destination $MAVEN_HOME_PARENT | Out-Null +} catch { + if (! (Test-Path -Path "$MAVEN_HOME" -PathType Container)) { + Write-Error "fail to move MAVEN_HOME" + } +} finally { + try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } + catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } +} + +Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" diff --git a/apache-seata/shop-service/pom.xml b/apache-seata/shop-service/pom.xml new file mode 100644 index 000000000000..2ca761ccd1f7 --- /dev/null +++ b/apache-seata/shop-service/pom.xml @@ -0,0 +1,59 @@ + + + 4.0.0 + com.baeldung + apache-seata-shop-service + 1.0.0-SNAPSHOT + apache-seata-shop-service + Apache Seata - Shop Service + + + org.springframework.boot + spring-boot-starter-parent + 3.5.11 + + + + + + org.springframework.boot + spring-boot-starter-jdbc + + + org.springframework.boot + spring-boot-starter-web + + + org.apache.seata + seata-spring-boot-starter + ${seata.version} + + + org.postgresql + postgresql + runtime + + + org.springframework.boot + spring-boot-starter-test + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + + + 17 + 2.6.0 + + + diff --git a/apache-seata/shop-service/src/main/java/com/baeldung/shop/ApacheSeataShopApplication.java b/apache-seata/shop-service/src/main/java/com/baeldung/shop/ApacheSeataShopApplication.java new file mode 100644 index 000000000000..e68ac4fceae7 --- /dev/null +++ b/apache-seata/shop-service/src/main/java/com/baeldung/shop/ApacheSeataShopApplication.java @@ -0,0 +1,13 @@ +package com.baeldung.shop; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +@SpringBootApplication +public class ApacheSeataShopApplication { + + public static void main(String[] args) { + SpringApplication.run(ApacheSeataShopApplication.class, args); + } + +} diff --git a/apache-seata/shop-service/src/main/java/com/baeldung/shop/Client.java b/apache-seata/shop-service/src/main/java/com/baeldung/shop/Client.java new file mode 100644 index 000000000000..111dd517ab85 --- /dev/null +++ b/apache-seata/shop-service/src/main/java/com/baeldung/shop/Client.java @@ -0,0 +1,26 @@ +package com.baeldung.shop; + +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.web.client.RestClient; + +public class Client { + private static final Logger LOG = LoggerFactory.getLogger(Client.class); + + private String url; + + private RestClient restClient; + + public Client(String url, RestClient restClient) { + this.url = url; + this.restClient = restClient; + } + + public void callService(String mode) { + LOG.info("Making request to {} with mode {}", url, mode); + var response = restClient.post().uri(url, Map.of("mode", mode)).retrieve(); + LOG.info("Made request. Response={}", response.toBodilessEntity()); + } +} diff --git a/apache-seata/shop-service/src/main/java/com/baeldung/shop/ClientConfiguration.java b/apache-seata/shop-service/src/main/java/com/baeldung/shop/ClientConfiguration.java new file mode 100644 index 000000000000..27e9041a710a --- /dev/null +++ b/apache-seata/shop-service/src/main/java/com/baeldung/shop/ClientConfiguration.java @@ -0,0 +1,28 @@ +package com.baeldung.shop; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.web.client.RestClient; + +@Configuration +public class ClientConfiguration { + + @Bean + public Client inventoryServiceClient(RestClient restClient, + @Value("${inventory_service.url}/inventory/{mode}") String url) { + return new Client(url, restClient); + } + + @Bean + public Client orderServiceClient(RestClient restClient, + @Value("${order_service.url}/order/{mode}") String url) { + return new Client(url, restClient); + } + + @Bean + public Client billingServiceClient(RestClient restClient, + @Value("${billing_service.url}/billing/{mode}") String url) { + return new Client(url, restClient); + } +} diff --git a/apache-seata/shop-service/src/main/java/com/baeldung/shop/Controller.java b/apache-seata/shop-service/src/main/java/com/baeldung/shop/Controller.java new file mode 100644 index 000000000000..e8a1ea139a79 --- /dev/null +++ b/apache-seata/shop-service/src/main/java/com/baeldung/shop/Controller.java @@ -0,0 +1,39 @@ +package com.baeldung.shop; + +import org.apache.seata.spring.annotation.GlobalTransactional; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RestController; + +@RestController +public class Controller { + @Autowired + private Repository repository; + + @Autowired + @Qualifier("inventoryServiceClient") + private Client inventoryServiceClient; + + @Autowired + @Qualifier("orderServiceClient") + private Client orderServiceClient; + + @Autowired + @Qualifier("billingServiceClient") + private Client billingServiceClient; + + @PostMapping("/shop/{mode}") + @GlobalTransactional + public void handle(@PathVariable("mode") String mode) { + repository.updateDatabase(); + inventoryServiceClient.callService(mode); + orderServiceClient.callService(mode); + billingServiceClient.callService(mode); + + if ("shop".equals(mode)) { + throw new RuntimeException("Shop Service failed"); + } + } +} diff --git a/apache-seata/shop-service/src/main/java/com/baeldung/shop/Repository.java b/apache-seata/shop-service/src/main/java/com/baeldung/shop/Repository.java new file mode 100644 index 000000000000..7792317726fa --- /dev/null +++ b/apache-seata/shop-service/src/main/java/com/baeldung/shop/Repository.java @@ -0,0 +1,34 @@ +package com.baeldung.shop; + +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Map; +import java.util.UUID; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; +import org.springframework.stereotype.Service; + +@Service +public class Repository { + private static final Logger LOG = LoggerFactory.getLogger(Repository.class); + + @Autowired + private NamedParameterJdbcTemplate jdbcTemplate; + + public void updateDatabase() { + var params = Map.of( + "id", UUID.randomUUID().toString(), + "created", Instant.now().atOffset(ZoneOffset.UTC) + ); + + LOG.info("Updating database with {}", params); + + int result = jdbcTemplate.update("INSERT INTO shop_table(id, created) VALUES (:id, :created)", + params); + + LOG.info("Updating database with result {}", result); + } +} diff --git a/apache-seata/shop-service/src/main/java/com/baeldung/shop/RestClientConfig.java b/apache-seata/shop-service/src/main/java/com/baeldung/shop/RestClientConfig.java new file mode 100644 index 000000000000..f36e3fb369fa --- /dev/null +++ b/apache-seata/shop-service/src/main/java/com/baeldung/shop/RestClientConfig.java @@ -0,0 +1,15 @@ +package com.baeldung.shop; + +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.web.client.RestClient; + +@Configuration +public class RestClientConfig { + @Bean + public RestClient restClient(SeataXidClientInterceptor seataXidClientInterceptor) { + return RestClient.builder() + .requestInterceptor(seataXidClientInterceptor) + .build(); + } +} diff --git a/apache-seata/shop-service/src/main/java/com/baeldung/shop/SeataXidClientInterceptor.java b/apache-seata/shop-service/src/main/java/com/baeldung/shop/SeataXidClientInterceptor.java new file mode 100644 index 000000000000..8cd67c085f49 --- /dev/null +++ b/apache-seata/shop-service/src/main/java/com/baeldung/shop/SeataXidClientInterceptor.java @@ -0,0 +1,33 @@ +package com.baeldung.shop; + +import java.io.IOException; + +import org.apache.seata.core.context.RootContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.http.HttpRequest; +import org.springframework.http.client.ClientHttpRequestExecution; +import org.springframework.http.client.ClientHttpRequestInterceptor; +import org.springframework.http.client.ClientHttpResponse; +import org.springframework.stereotype.Component; +import org.springframework.util.StringUtils; + +@Component +public class SeataXidClientInterceptor implements ClientHttpRequestInterceptor { + + private static final Logger LOG = LoggerFactory.getLogger(SeataXidClientInterceptor.class); + + @Override + public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) + throws IOException { + + String xid = RootContext.getXID(); + if (StringUtils.hasText(xid)) { + LOG.info("Propagating Seata XID: {}", xid); + + request.getHeaders().add(RootContext.KEY_XID, xid); + } + + return execution.execute(request, body); + } +} \ No newline at end of file diff --git a/apache-seata/shop-service/src/main/resources/application.properties b/apache-seata/shop-service/src/main/resources/application.properties new file mode 100644 index 000000000000..f47ac2caa59d --- /dev/null +++ b/apache-seata/shop-service/src/main/resources/application.properties @@ -0,0 +1,27 @@ +spring.application.name=apache-seata-a + +spring.datasource.generate-unique-name=false +spring.datasource.url=jdbc:postgresql://postgres:5432/seata +spring.datasource.username=seata +spring.datasource.password=seata +spring.datasource.driver-class-name=org.postgresql.Driver + +inventory_service.url=http://localhost:8081 +order_service.url=http://localhost:8082 +billing_service.url=http://localhost:8083 + +seata.enabled=true +seata.application-id=${spring.application.name} +seata.tx-service-group=my_tx_group + +seata.registry.type=file +seata.registry.file.name=seata.conf + +seata.config.type=file +seata.config.file.name=seata.conf + +seata.service.vgroup-mapping.my_tx_group=default +seata.service.grouplist.default=seata-server:8091 + +seata.data-source-proxy-mode=AT +seata.enable-auto-data-source-proxy=true diff --git a/apache-seata/shop-service/src/main/resources/seata.conf b/apache-seata/shop-service/src/main/resources/seata.conf new file mode 100644 index 000000000000..0219477600d9 --- /dev/null +++ b/apache-seata/shop-service/src/main/resources/seata.conf @@ -0,0 +1,60 @@ +transport { + type = "TCP" + server = "NIO" + heartbeat = true + thread-factory { + boss-thread-prefix = "NettyBoss" + worker-thread-prefix = "NettyServerNIOWorker" + server-executor-thread-size = 100 + share-boss-worker = false + client-selector-thread-size = 1 + client-selector-thread-prefix = "NettyClientSelector" + client-worker-thread-prefix = "NettyClientWorkerThread" + } + shutdown { + wait = 3 + } + serialization = "seata" + compressor = "none" +} + +service { + vgroupMapping.my_tx_group = "default" + default.grouplist = "seata-server:8091" + enableDegrade = false + disableGlobalTransaction = false +} + +client { + rm { + asyncCommitBufferLimit = 10000 + lock { + retryInterval = 10 + retryTimes = 30 + retryPolicyBranchRollbackOnConflict = true + } + reportRetryCount = 5 + tableMetaCheckEnable = false + reportSuccessEnable = false + sagaBranchRegisterEnable = false + } + tm { + commitRetryCount = 5 + rollbackRetryCount = 5 + defaultGlobalTransactionTimeout = 60000 + degradeCheck = false + } + undo { + dataValidation = true + logSerialization = "jackson" + logTable = "undo_log" + compress { + enable = true + type = "zip" + threshold = "64k" + } + } + log { + exceptionRate = 100 + } +} \ No newline at end of file diff --git a/apache-seata/sql/schema.sql b/apache-seata/sql/schema.sql new file mode 100644 index 000000000000..d7deb9e48451 --- /dev/null +++ b/apache-seata/sql/schema.sql @@ -0,0 +1,19 @@ +CREATE TABLE shop_table ( + id TEXT PRIMARY KEY, + created TIMESTAMP NOT NULL +); + +CREATE TABLE inventory_table ( + id TEXT PRIMARY KEY, + created TIMESTAMP NOT NULL +); + +CREATE TABLE order_table ( + id TEXT PRIMARY KEY, + created TIMESTAMP NOT NULL +); + +CREATE TABLE billing_table ( + id TEXT PRIMARY KEY, + created TIMESTAMP NOT NULL +); diff --git a/apache-seata/sql/seata.sql b/apache-seata/sql/seata.sql new file mode 100644 index 000000000000..57d1bf0e852c --- /dev/null +++ b/apache-seata/sql/seata.sql @@ -0,0 +1,12 @@ +CREATE TABLE IF NOT EXISTS undo_log ( + id BIGSERIAL NOT NULL, + branch_id BIGINT NOT NULL, + xid VARCHAR(128) NOT NULL, + context VARCHAR(128) NOT NULL, + rollback_info BYTEA NOT NULL, + log_status INT NOT NULL, + log_created TIMESTAMP(0) NOT NULL, + log_modified TIMESTAMP(0) NOT NULL, + CONSTRAINT pk_undo_log PRIMARY KEY (id), + CONSTRAINT ux_undo_log UNIQUE (xid, branch_id) +); \ No newline at end of file diff --git a/apache-spark-2/pom.xml b/apache-spark-2/pom.xml new file mode 100644 index 000000000000..d9bfe5b26768 --- /dev/null +++ b/apache-spark-2/pom.xml @@ -0,0 +1,71 @@ + + + 4.0.0 + apache-spark-2 + 1.0-SNAPSHOT + jar + apache-spark-2 + + + com.baeldung + parent-modules + 1.0.0-SNAPSHOT + + + + + io.delta + delta-core_2.12 + ${delta-core.version} + + + org.apache.spark + spark-core_2.12 + ${org.apache.spark.spark-core.version} + + + org.apache.spark + spark-sql_2.12 + ${org.apache.spark.spark-sql.version} + + + + + + + maven-assembly-plugin + ${maven-assembly-plugin.version} + + + package + + single + + + + + + jar-with-dependencies + + + + + + + + + SparkPackagesRepo + https://repos.spark-packages.org + + + + + 2.4.0 + 3.4.0 + 3.4.0 + 3.3.0 + + + diff --git a/apache-spark-2/src/main/java/com/baeldung/delta/DeltaLake.java b/apache-spark-2/src/main/java/com/baeldung/delta/DeltaLake.java new file mode 100644 index 000000000000..cf1a3fed7b33 --- /dev/null +++ b/apache-spark-2/src/main/java/com/baeldung/delta/DeltaLake.java @@ -0,0 +1,60 @@ +package com.baeldung.delta; + +import org.apache.spark.sql.*; +import java.io.Serializable; +import java.nio.file.Files; + +public class DeltaLake { + public static SparkSession createSession() { + return SparkSession.builder() + .appName("DeltaLake") + .master("local[*]") + .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension") + .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog") + .getOrCreate(); + } + + public static String preparePeopleTable(SparkSession spark) { + try { + String tablePath = Files.createTempDirectory("delta-table-").toAbsolutePath().toString(); + + Dataset data = spark.createDataFrame( + java.util.Arrays.asList( + new Person(1, "Alice"), + new Person(2, "Bob") + ), + Person.class + ); + + data.write().format("delta").mode("overwrite").save(tablePath); + spark.sql("DROP TABLE IF EXISTS people"); + spark.sql("CREATE TABLE IF NOT EXISTS people USING DELTA LOCATION '" + tablePath + "'"); + return tablePath; + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + public static void cleanupPeopleTable(SparkSession spark) { + spark.sql("DROP TABLE IF EXISTS people"); + } + + public static void stopSession(SparkSession spark) { + if (spark != null) { + spark.stop(); + } + } + + public static class Person implements Serializable { + private int id; + private String name; + + public Person() {} + public Person(int id, String name) { this.id = id; this.name = name; } + + public int getId() { return id; } + public void setId(int id) { this.id = id; } + public String getName() { return name; } + public void setName(String name) { this.name = name; } + } +} diff --git a/apache-spark-2/src/test/java/com/baeldung/delta/DeltaLakeUnitTest.java b/apache-spark-2/src/test/java/com/baeldung/delta/DeltaLakeUnitTest.java new file mode 100644 index 000000000000..68ee84bc6e3a --- /dev/null +++ b/apache-spark-2/src/test/java/com/baeldung/delta/DeltaLakeUnitTest.java @@ -0,0 +1,43 @@ +package com.baeldung.delta; + +import org.apache.spark.sql.Dataset; +import org.apache.spark.sql.Row; +import org.apache.spark.sql.SparkSession; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class DeltaLakeUnitTest { + + private static SparkSession spark; + private static String tablePath; + + @BeforeAll + static void setUp() { + spark = DeltaLake.createSession(); + tablePath = DeltaLake.preparePeopleTable(spark); + } + + @AfterAll + static void tearDown() { + try { + DeltaLake.cleanupPeopleTable(spark); + } finally { + DeltaLake.stopSession(spark); + } + } + + @Test + void givenDeltaLake_whenUsingDeltaFormat_thenPrintAndValidate() { + Dataset df = spark.sql("DESCRIBE DETAIL people"); + df.show(false); + + Row row = df.first(); + assertEquals("file:"+tablePath, row.getAs("location")); + assertEquals("delta", row.getAs("format")); + assertTrue(row.getAs("numFiles") >= 1); + } +} diff --git a/apache-spark-2/src/test/java/com/baeldung/delta/SparkExecutorConfigurationUnitTest.java b/apache-spark-2/src/test/java/com/baeldung/delta/SparkExecutorConfigurationUnitTest.java new file mode 100644 index 000000000000..2c262f7dceb0 --- /dev/null +++ b/apache-spark-2/src/test/java/com/baeldung/delta/SparkExecutorConfigurationUnitTest.java @@ -0,0 +1,42 @@ +package com.baeldung.delta; + +import org.apache.spark.SparkConf; +import org.apache.spark.sql.SparkSession; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +public class SparkExecutorConfigurationUnitTest { + + private static SparkSession spark; + + @BeforeAll + public static void setUp() { + spark = SparkSession.builder() + .appName("StaticExecutorAllocationExample") + .config("spark.executor.instances", "8") + .config("spark.executor.cores", "4") + .config("spark.executor.memory", "8G") + .master("local[*]") + .getOrCreate(); + } + + @AfterAll + public static void tearDown() { + if (spark != null) { + spark.stop(); + } + } + + @Test + public void givenExecutor_whenUsingStaticAllocation_thenPrintAndValidate() { + SparkConf conf = spark.sparkContext().getConf(); + + assertEquals("8", conf.get("spark.executor.instances")); + assertEquals("4", conf.get("spark.executor.cores")); + assertEquals("8G", conf.get("spark.executor.memory")); + assertEquals("StaticExecutorAllocationExample", conf.get("spark.app.name")); + } +} diff --git a/apache-spark/README.md b/apache-spark/README.md deleted file mode 100644 index 862626988b08..000000000000 --- a/apache-spark/README.md +++ /dev/null @@ -1,12 +0,0 @@ -## Apache Spark - -This module contains articles about Apache Spark - -### Relevant articles: - -- [Introduction to Apache Spark](https://www.baeldung.com/apache-spark) -- [Building a Data Pipeline with Kafka, Spark Streaming and Cassandra](https://www.baeldung.com/kafka-spark-data-pipeline) -- [Machine Learning with Spark MLlib](https://www.baeldung.com/spark-mlib-machine-learning) -- [Introduction to Spark Graph Processing with GraphFrames](https://www.baeldung.com/spark-graph-graphframes) -- [Apache Spark: Differences between Dataframes, Datasets and RDDs](https://www.baeldung.com/java-spark-dataframe-dataset-rdd) -- [Spark DataFrame](https://www.baeldung.com/spark-dataframes) diff --git a/apache-spark/src/main/java/com/baeldung/spark/dataframeconcat/ConcatRowsExample.java b/apache-spark/src/main/java/com/baeldung/spark/dataframeconcat/ConcatRowsExample.java new file mode 100644 index 000000000000..7dca8c658813 --- /dev/null +++ b/apache-spark/src/main/java/com/baeldung/spark/dataframeconcat/ConcatRowsExample.java @@ -0,0 +1,89 @@ +package com.baeldung.spark.dataframeconcat; + +import org.apache.spark.sql.Dataset; +import org.apache.spark.sql.Row; +import org.apache.spark.sql.SparkSession; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Arrays; +import java.util.List; + +public class ConcatRowsExample { + + private static final Logger logger = LoggerFactory.getLogger(ConcatRowsExample.class); + + public static void main(String[] args) { + SparkSession spark = SparkSession.builder() + .appName("Row-wise Concatenation Example") + .master("local[*]") + .getOrCreate(); + + try { + // Create sample data + List data1 = Arrays.asList( + new Person(1, "Alice"), + new Person(2, "Bob") + ); + + List data2 = Arrays.asList( + new Person(3, "Charlie"), + new Person(4, "Diana") + ); + + Dataset df1 = spark.createDataFrame(data1, Person.class); + Dataset df2 = spark.createDataFrame(data2, Person.class); + + logger.info("First DataFrame:"); + df1.show(); + + logger.info("Second DataFrame:"); + df2.show(); + + // Row-wise concatenation using reusable method + Dataset combined = concatenateDataFrames(df1, df2); + + logger.info("After row-wise concatenation:"); + combined.show(); + } finally { + spark.stop(); + } + } + + /** + * Concatenates two DataFrames row-wise using unionByName. + * This method is extracted for reusability and testing. + */ + public static Dataset concatenateDataFrames(Dataset df1, Dataset df2) { + return df1.unionByName(df2); + } + + public static class Person implements java.io.Serializable { + private int id; + private String name; + + public Person() { + } + + public Person(int id, String name) { + this.id = id; + this.name = name; + } + + public int getId() { + return id; + } + + public void setId(int id) { + this.id = id; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + } +} diff --git a/apache-spark/src/test/java/com/baeldung/spark/dataframeconcat/ConcatRowsExampleUnitTest.java b/apache-spark/src/test/java/com/baeldung/spark/dataframeconcat/ConcatRowsExampleUnitTest.java new file mode 100644 index 000000000000..8db730240499 --- /dev/null +++ b/apache-spark/src/test/java/com/baeldung/spark/dataframeconcat/ConcatRowsExampleUnitTest.java @@ -0,0 +1,83 @@ +package com.baeldung.spark.dataframeconcat; + +import org.apache.spark.sql.Dataset; +import org.apache.spark.sql.Row; +import org.apache.spark.sql.SparkSession; +import org.junit.jupiter.api.*; + +import java.util.Arrays; + +import static org.junit.jupiter.api.Assertions.*; + +class ConcatRowsExampleUnitTest { + + private static SparkSession spark; + private Dataset df1; + private Dataset df2; + + @BeforeAll + static void setupClass() { + spark = SparkSession.builder() + .appName("Row-wise Concatenation Test") + .master("local[*]") + .getOrCreate(); + } + + @BeforeEach + void setup() { + df1 = spark.createDataFrame( + Arrays.asList( + new ConcatRowsExample.Person(1, "Alice"), + new ConcatRowsExample.Person(2, "Bob") + ), + ConcatRowsExample.Person.class + ); + + df2 = spark.createDataFrame( + Arrays.asList( + new ConcatRowsExample.Person(3, "Charlie"), + new ConcatRowsExample.Person(4, "Diana") + ), + ConcatRowsExample.Person.class + ); + } + + @AfterAll + static void tearDownClass() { + spark.stop(); + } + + @Test + void givenTwoDataFrames_whenConcatenated_thenRowCountMatches() { + Dataset combined = ConcatRowsExample.concatenateDataFrames(df1, df2); + + assertEquals( + 4, + combined.count(), + "The combined DataFrame should have 4 rows" + ); + } + + @Test + void givenTwoDataFrames_whenConcatenated_thenSchemaRemainsSame() { + Dataset combined = ConcatRowsExample.concatenateDataFrames(df1, df2); + + assertEquals( + df1.schema(), + combined.schema(), + "Schema should remain consistent after concatenation" + ); + } + + @Test + void givenTwoDataFrames_whenConcatenated_thenDataContainsExpectedName() { + Dataset combined = ConcatRowsExample.concatenateDataFrames(df1, df2); + + assertTrue( + combined + .filter("name = 'Charlie'") + .count() > 0, + "Combined DataFrame should contain Charlie" + ); + } +} diff --git a/apache-thrift/README.md b/apache-thrift/README.md deleted file mode 100644 index 4508939de64a..000000000000 --- a/apache-thrift/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Apache Thrift - -This module contains articles about Apache Thrift - -### Relevant articles: - -- [Working with Apache Thrift](https://www.baeldung.com/apache-thrift) diff --git a/apache-velocity/README.md b/apache-velocity/README.md deleted file mode 100644 index d539d79efc57..000000000000 --- a/apache-velocity/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Apache Velocity - -This module contains articles about Apache Velocity - -### Relevant articles: - -- [Introduction to Apache Velocity](https://www.baeldung.com/apache-velocity) diff --git a/apollo/pom.xml b/apollo/pom.xml new file mode 100644 index 000000000000..e7b7abbc7bf3 --- /dev/null +++ b/apollo/pom.xml @@ -0,0 +1,44 @@ + + + 4.0.0 + apollo + 1.0-SNAPSHOT + jar + apollo + http://maven.apache.org + + + com.baeldung + parent-boot-3 + 0.0.1-SNAPSHOT + ../parent-boot-3 + + + + + org.springframework.boot + spring-boot-starter-web + + + com.ctrip.framework.apollo + apollo-client + ${apollo.version} + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + + + 2.4.0 + + + diff --git a/apollo/src/main/java/com/example/demo/DemoApplication.java b/apollo/src/main/java/com/example/demo/DemoApplication.java new file mode 100644 index 000000000000..4b96628d2ba5 --- /dev/null +++ b/apollo/src/main/java/com/example/demo/DemoApplication.java @@ -0,0 +1,16 @@ +package com.example.demo; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +import com.ctrip.framework.apollo.spring.annotation.EnableApolloConfig; + +@SpringBootApplication +@EnableApolloConfig +public class DemoApplication { + + public static void main(String[] args) { + SpringApplication.run(DemoApplication.class, args); + } + +} diff --git a/apollo/src/main/java/com/example/demo/DemoController.java b/apollo/src/main/java/com/example/demo/DemoController.java new file mode 100644 index 000000000000..05769b0aa2b9 --- /dev/null +++ b/apollo/src/main/java/com/example/demo/DemoController.java @@ -0,0 +1,20 @@ +package com.example.demo; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RestController; +import org.springframework.web.bind.annotation.RequestMapping; + +@RestController +@RequestMapping("/config") +public class DemoController { + // We're not using JDBC, but this is the example property from the article. + @Value("${spring.datasource.url}") + private String someValue; + + @GetMapping + public String get() { + return "Hello, " + someValue; + } +} + diff --git a/apollo/src/main/resources/application.properties b/apollo/src/main/resources/application.properties new file mode 100644 index 000000000000..096e33eec500 --- /dev/null +++ b/apollo/src/main/resources/application.properties @@ -0,0 +1,7 @@ +spring.application.name=demo + +server.port=9090 + +app.id=baeldung-test +apollo.cluster=emea +apollo.meta=http://localhost:8080 diff --git a/aspectj/pom.xml b/aspectj/pom.xml index 7510ee50ed7b..db6e26c94d65 100644 --- a/aspectj/pom.xml +++ b/aspectj/pom.xml @@ -1,26 +1,25 @@ - + 4.0.0 + aspectj + 0.0.1-SNAPSHOT + aspectj + Demo project for Spring Boot to Pointcut all methods in a certain package + com.baeldung parent-boot-3 0.0.1-SNAPSHOT ../parent-boot-3 - aspectj - 0.0.1-SNAPSHOT - aspectj - Demo project for Spring Boot to Pointcut all methods in a certain package - - 17 - + org.springframework.boot spring-boot-starter - org.aspectj aspectjrt @@ -31,7 +30,6 @@ aspectjweaver 1.9.22.1 - org.springframework.boot spring-boot-starter-test @@ -57,4 +55,8 @@ + + 17 + + diff --git a/atomix/README.md b/atomix/README.md deleted file mode 100644 index 217fced82a80..000000000000 --- a/atomix/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Atomix - -This module contains articles about Atomix - -### Relevant articles: - -- [Introduction to Atomix](https://www.baeldung.com/atomix) diff --git a/aws-modules/amazon-athena/README.md b/aws-modules/amazon-athena/README.md deleted file mode 100644 index f27f17cd3e55..000000000000 --- a/aws-modules/amazon-athena/README.md +++ /dev/null @@ -1,2 +0,0 @@ -### Relevant Articles -- [Using Amazon Athena With Spring Boot to Query S3 Data](https://www.baeldung.com/spring-boot-amazon-athena) diff --git a/aws-modules/amazon-athena/pom.xml b/aws-modules/amazon-athena/pom.xml index 059864b3f2aa..0e91add9eac9 100644 --- a/aws-modules/amazon-athena/pom.xml +++ b/aws-modules/amazon-athena/pom.xml @@ -1,81 +1,91 @@ - 4.0.0 - amazon-athena - 0.0.1 - jar - amazon-athena - codebase demonstrating the integration of Amazon Athena in Spring Boot to query data stored in a S3 bucket + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> + 4.0.0 + amazon-athena + 0.0.1 + jar + amazon-athena + codebase demonstrating the integration of Amazon Athena in Spring Boot to query data stored in a S3 bucket - - com.baeldung - parent-boot-3 - 0.0.1-SNAPSHOT - ../../parent-boot-3 - + + com.baeldung + aws-modules + 1.0.0-SNAPSHOT + - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-validation - - - org.springframework.boot - spring-boot-configuration-processor - - - software.amazon.awssdk - athena - ${amazon-athena.version} - - - commons-io - commons-io - ${commons-io.version} - - - org.json - json - ${org-json.version} - - - com.fasterxml.jackson.datatype - jackson-datatype-json-org - - - org.projectlombok - lombok - true - - + + + + org.springframework.boot + spring-boot-dependencies + ${spring-boot.version} + pom + import + + + - - - - org.springframework.boot - spring-boot-maven-plugin - - - - org.projectlombok - lombok - - - - - - + + + org.springframework.boot + spring-boot-starter-web + + + org.springframework.boot + spring-boot-starter-validation + + + org.springframework.boot + spring-boot-configuration-processor + + + software.amazon.awssdk + athena + + + commons-io + commons-io + ${commons-io.version} + + + org.json + json + ${org-json.version} + + + com.fasterxml.jackson.datatype + jackson-datatype-json-org + + + org.projectlombok + lombok + true + + - - 20240303 - 2.16.1 - 2.26.0 - + + + + org.springframework.boot + spring-boot-maven-plugin + ${spring-boot.version} + + + + org.projectlombok + lombok + + + + + + + + + 20240303 + 2.16.1 + \ No newline at end of file diff --git a/aws-modules/amazon-textract/pom.xml b/aws-modules/amazon-textract/pom.xml index efe6056dd835..7ca20118274b 100644 --- a/aws-modules/amazon-textract/pom.xml +++ b/aws-modules/amazon-textract/pom.xml @@ -11,11 +11,22 @@ com.baeldung - parent-boot-3 - 0.0.1-SNAPSHOT - ../../parent-boot-3 + aws-modules + 1.0.0-SNAPSHOT + + + + org.springframework.boot + spring-boot-dependencies + ${spring-boot.version} + pom + import + + + + org.springframework.boot @@ -29,15 +40,14 @@ org.springframework.boot spring-boot-configuration-processor + + org.springframework.boot + spring-boot-starter-test + software.amazon.awssdk textract - ${amazon-textract.version} - - 2.27.5 - - \ No newline at end of file diff --git a/aws-modules/aws-app-sync/README.md b/aws-modules/aws-app-sync/README.md deleted file mode 100644 index 976a999f409e..000000000000 --- a/aws-modules/aws-app-sync/README.md +++ /dev/null @@ -1,3 +0,0 @@ -### Relevant Articles: - -- [AWS AppSync With Spring Boot](https://www.baeldung.com/aws-appsync-spring) diff --git a/aws-modules/aws-app-sync/pom.xml b/aws-modules/aws-app-sync/pom.xml index 8d7d90e63177..a46bd33908d4 100644 --- a/aws-modules/aws-app-sync/pom.xml +++ b/aws-modules/aws-app-sync/pom.xml @@ -9,19 +9,20 @@ com.baeldung - parent-boot-2 - 0.0.1-SNAPSHOT - ../../parent-boot-2 + aws-modules + 1.0.0-SNAPSHOT org.springframework.boot spring-boot-starter-web + ${spring-boot.version} org.springframework.boot spring-boot-starter-test + ${spring-boot.version} test @@ -33,6 +34,7 @@ org.springframework.boot spring-boot-starter-webflux + ${spring-boot.version} @@ -41,8 +43,13 @@ org.springframework.boot spring-boot-maven-plugin + ${spring-boot.version} + + 3.3.2 + + \ No newline at end of file diff --git a/aws-modules/aws-dynamodb-v2/.gitignore b/aws-modules/aws-dynamodb-v2/.gitignore new file mode 100644 index 000000000000..bf11a4cc38c4 --- /dev/null +++ b/aws-modules/aws-dynamodb-v2/.gitignore @@ -0,0 +1,2 @@ +/target/ +.idea/ \ No newline at end of file diff --git a/aws-modules/aws-dynamodb-v2/pom.xml b/aws-modules/aws-dynamodb-v2/pom.xml new file mode 100644 index 000000000000..e3d3d3d512b1 --- /dev/null +++ b/aws-modules/aws-dynamodb-v2/pom.xml @@ -0,0 +1,106 @@ + + + 4.0.0 + aws-dynamodb-v2 + 0.1.0-SNAPSHOT + jar + aws-dynamodb-v2 + + + com.baeldung + aws-modules + 1.0.0-SNAPSHOT + + + + + software.amazon.awssdk + dynamodb + ${software.amazon.awssdk.version} + + + org.testcontainers + localstack + ${testcontainers.version} + test + + + org.testcontainers + testcontainers + ${testcontainers.version} + test + + + software.amazon.awssdk + sdk-core + ${software.amazon.awssdk.version} + + + software.amazon.awssdk + aws-core + ${software.amazon.awssdk.version} + + + software.amazon.awssdk + netty-nio-client + ${software.amazon.awssdk.version} + + + software.amazon.awssdk + utils + ${software.amazon.awssdk.version} + + + software.amazon.awssdk + identity-spi + ${software.amazon.awssdk.version} + + + software.amazon.awssdk + checksums + ${software.amazon.awssdk.version} + + + + + + + org.apache.maven.plugins + maven-dependency-plugin + ${maven-plugins-version} + + + copy + compile + + copy-dependencies + + + + so,dll,dylib + native-libs + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + 9 + 9 + 9 + + + + + + + 3.1.1 + 2.31.23 + 1.20.6 + + + \ No newline at end of file diff --git a/aws-modules/aws-dynamodb-v2/src/main/java/com/baeldung/dynamodb/query/UserOrdersRepository.java b/aws-modules/aws-dynamodb-v2/src/main/java/com/baeldung/dynamodb/query/UserOrdersRepository.java new file mode 100644 index 000000000000..b62ac3e928b0 --- /dev/null +++ b/aws-modules/aws-dynamodb-v2/src/main/java/com/baeldung/dynamodb/query/UserOrdersRepository.java @@ -0,0 +1,99 @@ +package com.baeldung.dynamodb.query; +import software.amazon.awssdk.services.dynamodb.DynamoDbClient; +import software.amazon.awssdk.services.dynamodb.model.*; + +import java.util.*; + +public class UserOrdersRepository { + + private final DynamoDbClient dynamoDb; + private static final String TABLE_NAME = "UserOrders"; + + public UserOrdersRepository(DynamoDbClient dynamoDb) { + this.dynamoDb = dynamoDb; + } + + public List> getOrdersByUserId(String userId) { + QueryRequest request = QueryRequest.builder() + .tableName(TABLE_NAME) + .keyConditionExpression("userId = :uid") + .expressionAttributeValues(Map.of( + ":uid", AttributeValue.fromS(userId) + )) + .build(); + + QueryResponse response = dynamoDb.query(request); + + List> items = response.items(); + + for (Map item : items) { + System.out.println("Order item: " + item.get("item").s()); + } + + return response.items(); + } + + public List> getOrdersAfterDate(String userId, String startDate) { + QueryRequest request = QueryRequest.builder() + .tableName(TABLE_NAME) + .keyConditionExpression("userId = :uid AND orderDate > :startDate") + .expressionAttributeValues(Map.of( + ":uid", AttributeValue.fromS(userId), + ":startDate", AttributeValue.fromS(startDate) + )) + .build(); + + return dynamoDb.query(request).items(); + } + + public List> getOrdersBetweenDates(String userId, String fromDate, String toDate) { + QueryRequest request = QueryRequest.builder() + .tableName(TABLE_NAME) + .keyConditionExpression("userId = :uid AND orderDate BETWEEN :from AND :to") + .expressionAttributeValues(Map.of( + ":uid", AttributeValue.fromS(userId), + ":from", AttributeValue.fromS(fromDate), + ":to", AttributeValue.fromS(toDate) + )) + .build(); + + return dynamoDb.query(request).items(); + } + + public List> getOrdersByMonth(String userId, String monthPrefix) { + QueryRequest request = QueryRequest.builder() + .tableName(TABLE_NAME) + .keyConditionExpression("userId = :uid AND begins_with(orderDate, :prefix)") + .expressionAttributeValues(Map.of( + ":uid", AttributeValue.fromS(userId), + ":prefix", AttributeValue.fromS(monthPrefix) + )) + .build(); + + return dynamoDb.query(request).items(); + } + + public List> getAllOrdersPaginated(String userId) { + List> allItems = new ArrayList<>(); + Map lastKey = null; + + do { + QueryRequest.Builder requestBuilder = QueryRequest.builder() + .tableName("UserOrders") + .keyConditionExpression("userId = :uid") + .expressionAttributeValues(Map.of( + ":uid", AttributeValue.fromS(userId) + )); + + if (lastKey != null) { + requestBuilder.exclusiveStartKey(lastKey); + } + + QueryResponse response = dynamoDb.query(requestBuilder.build()); + allItems.addAll(response.items()); + lastKey = response.lastEvaluatedKey(); + } while (lastKey != null && !lastKey.isEmpty()); + + return allItems; + } +} diff --git a/aws-modules/aws-dynamodb-v2/src/test/java/com/baeldung/dynamodb/query/UserOrdersRepositoryLiveTest.java b/aws-modules/aws-dynamodb-v2/src/test/java/com/baeldung/dynamodb/query/UserOrdersRepositoryLiveTest.java new file mode 100644 index 000000000000..92cb4fcd0d7e --- /dev/null +++ b/aws-modules/aws-dynamodb-v2/src/test/java/com/baeldung/dynamodb/query/UserOrdersRepositoryLiveTest.java @@ -0,0 +1,130 @@ +package com.baeldung.dynamodb.query; + +import org.junit.jupiter.api.*; +import org.testcontainers.containers.localstack.LocalStackContainer; +import org.testcontainers.utility.DockerImageName; +import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; +import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; +import software.amazon.awssdk.regions.Region; +import software.amazon.awssdk.services.dynamodb.DynamoDbClient; +import software.amazon.awssdk.services.dynamodb.model.*; + +import java.util.*; +import java.util.stream.Collectors; + +import static org.junit.jupiter.api.Assertions.*; + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class UserOrdersRepositoryLiveTest { + + private static final String TABLE_NAME = "UserOrders"; + private DynamoDbClient dynamoDb; + private UserOrdersRepository repository; + + private final LocalStackContainer localstack = new LocalStackContainer( + DockerImageName.parse("localstack/localstack:latest")) + .withServices(LocalStackContainer.Service.DYNAMODB); + + @BeforeAll + void setUp() { + localstack.start(); + + dynamoDb = DynamoDbClient.builder() + .endpointOverride(localstack.getEndpointOverride(LocalStackContainer.Service.DYNAMODB)) + .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("test", "test"))) + .region(Region.of(localstack.getRegion())) + .build(); + + repository = new UserOrdersRepository(dynamoDb); + + createTable(); + seedData(); + } + + void createTable() { + dynamoDb.createTable(CreateTableRequest.builder() + .tableName(TABLE_NAME) + .keySchema( + KeySchemaElement.builder().attributeName("userId").keyType(KeyType.HASH).build(), + KeySchemaElement.builder().attributeName("orderDate").keyType(KeyType.RANGE).build() + ) + .attributeDefinitions( + AttributeDefinition.builder().attributeName("userId").attributeType(ScalarAttributeType.S).build(), + AttributeDefinition.builder().attributeName("orderDate").attributeType(ScalarAttributeType.S).build() + ) + .provisionedThroughput( + ProvisionedThroughput.builder().readCapacityUnits(5L).writeCapacityUnits(5L).build() + ) + .build()); + + dynamoDb.waiter().waitUntilTableExists(r -> r.tableName(TABLE_NAME)); + } + + void seedData() { + putOrder("2024-12-01", "Laptop"); + putOrder("2024-12-15", "Monitor"); + putOrder("2025-01-05", "Mouse"); + putOrder("2025-01-20", "Keyboard"); + } + + void putOrder(String orderDate, String itemName) { + dynamoDb.putItem(PutItemRequest.builder() + .tableName(TABLE_NAME) + .item(Map.of( + "userId", AttributeValue.fromS("user1"), + "orderDate", AttributeValue.fromS(orderDate), + "item", AttributeValue.fromS(itemName) + )) + .build()); + } + + @Test + void givenUserId_whenGetOrdersByUserId_thenReturnAllUserOrders() { + List> items = repository.getOrdersByUserId("user1"); + assertEquals(4, items.size()); + } + + @Test + void givenStartDate_whenGetOrdersAfterDate_thenReturnOnlyNewerOrders() { + List> items = repository.getOrdersAfterDate("user1", "2025-01-01"); + List names = items.stream().map(i -> i.get("item").s()).collect(Collectors.toList()); + + assertEquals(List.of("Mouse", "Keyboard"), names); + } + + @Test + void givenDateRange_whenGetOrdersBetweenDates_thenReturnOrdersInRange() { + List> items = repository.getOrdersBetweenDates("user1", "2024-12-01", "2024-12-31"); + List names = items.stream().map(i -> i.get("item").s()).collect(Collectors.toList()); + + assertEquals(List.of("Laptop", "Monitor"), names); + } + + @Test + void givenMonthPrefix_whenGetOrdersByMonth_thenReturnMonthlyOrders() { + List> items = repository.getOrdersByMonth("user1", "2025-01"); + List names = items.stream().map(i -> i.get("item").s()).collect(Collectors.toList()); + + assertEquals(List.of("Mouse", "Keyboard"), names); + } + + @Test + void givenUserId_whenGetAllOrdersPaginated_thenReturnAllUserOrders() { + List> items = repository.getAllOrdersPaginated("user1"); + + assertEquals(4, items.size()); + + List itemNames = items.stream() + .map(item -> item.get("item").s()) + .collect(Collectors.toList()); + + assertTrue(itemNames.containsAll(List.of("Laptop", "Monitor", "Mouse", "Keyboard"))); + } + + @AfterAll + void tearDown() { + if (localstack != null) { + localstack.stop(); + } + } +} \ No newline at end of file diff --git a/aws-modules/aws-dynamodb/README.md b/aws-modules/aws-dynamodb/README.md deleted file mode 100644 index 68a353e55566..000000000000 --- a/aws-modules/aws-dynamodb/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## AWS DYNAMODB - -This module contains articles about AWS DynamoDB - -### Relevant articles -- [Integration Testing with a Local DynamoDB Instance](https://www.baeldung.com/dynamodb-local-integration-tests) - diff --git a/aws-modules/aws-dynamodb/pom.xml b/aws-modules/aws-dynamodb/pom.xml index 8ad984174978..af0b8332e8db 100644 --- a/aws-modules/aws-dynamodb/pom.xml +++ b/aws-modules/aws-dynamodb/pom.xml @@ -63,6 +63,7 @@ + 1.12.331 2.11.0 1.21.1 3.1.1 diff --git a/aws-modules/aws-lambda-modules/README.md b/aws-modules/aws-lambda-modules/README.md deleted file mode 100644 index a845c5835e99..000000000000 --- a/aws-modules/aws-lambda-modules/README.md +++ /dev/null @@ -1,11 +0,0 @@ -## AWS Lambda - -This module contains articles about AWS Lambda - -### Relevant Articles: -- [A Basic AWS Lambda Example With Java](https://www.baeldung.com/java-aws-lambda) -- [Using AWS Lambda with API Gateway](https://www.baeldung.com/aws-lambda-api-gateway) -- [Introduction to AWS Serverless Application Model](https://www.baeldung.com/aws-serverless) -- [How to Implement Hibernate in an AWS Lambda Function in Java](https://www.baeldung.com/java-aws-lambda-hibernate) -- [Writing an Enterprise-Grade AWS Lambda in Java](https://www.baeldung.com/java-enterprise-aws-lambda) -- [AWS Lambda Using DynamoDB With Java](https://www.baeldung.com/aws-lambda-dynamodb-java) diff --git a/aws-modules/aws-lambda-modules/lambda-function/pom.xml b/aws-modules/aws-lambda-modules/lambda-function/pom.xml index 31b7a6c3e682..a415f1c66321 100644 --- a/aws-modules/aws-lambda-modules/lambda-function/pom.xml +++ b/aws-modules/aws-lambda-modules/lambda-function/pom.xml @@ -4,9 +4,9 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 lambda-function - lambda-function 0.1.0-SNAPSHOT jar + lambda-function com.baeldung @@ -24,6 +24,20 @@ com.amazonaws aws-java-sdk-core ${aws-java-sdk.version} + + + commons-logging + commons-logging + + + commons-codec + commons-codec + + + com.amazonaws + aws-java-sdk-pom + + com.amazonaws @@ -85,6 +99,23 @@ ${maven-shade-plugin.version} false + + + *:* + + + **/module-info.class + + META-INF/MANIFEST.MF + META-INF/LICENSE + META-INF/NOTICE + META-INF/LICENSE.txt + META-INF/NOTICE.txt + META-INF/services/com.fasterxml.jackson.core.JsonFactory + META-INF/DEPENDENCIES + + + diff --git a/aws-modules/aws-lambda-modules/shipping-tracker-lambda/ShippingFunction/pom.xml b/aws-modules/aws-lambda-modules/shipping-tracker-lambda/ShippingFunction/pom.xml index 0f294fabeb25..9fefdae9345e 100644 --- a/aws-modules/aws-lambda-modules/shipping-tracker-lambda/ShippingFunction/pom.xml +++ b/aws-modules/aws-lambda-modules/shipping-tracker-lambda/ShippingFunction/pom.xml @@ -3,9 +3,9 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> 4.0.0 ShippingFunction - ShippingFunction 1.0 jar + ShippingFunction com.baeldung @@ -45,7 +45,6 @@ HikariCP ${hikari.cp.version} - org.postgresql postgresql @@ -58,8 +57,26 @@ org.apache.maven.plugins maven-shade-plugin - 3.1.1 + ${maven-shade-plugin.version} + false + + + *:* + + **/module-info.class + META-INF/LICENSE + META-INF/LICENSE.md + META-INF/LICENSE.txt + META-INF/NOTICE + META-INF/NOTICE.md + META-INF/NOTICE.txt + META-INF/MANIFEST.MF + META-INF/DEPENDENCIES + META-INF/services/com.fasterxml.jackson.core.JsonFactory + + + @@ -76,6 +93,7 @@ 11 11 + 3.6.0 6.4.2.Final 1.2.0 3.1.0 diff --git a/aws-modules/aws-lambda-modules/todo-reminder-lambda/ToDoFunction/pom.xml b/aws-modules/aws-lambda-modules/todo-reminder-lambda/ToDoFunction/pom.xml index 458a41f7562d..beaaf066b753 100644 --- a/aws-modules/aws-lambda-modules/todo-reminder-lambda/ToDoFunction/pom.xml +++ b/aws-modules/aws-lambda-modules/todo-reminder-lambda/ToDoFunction/pom.xml @@ -4,9 +4,9 @@ 4.0.0 helloworld ToDoFunction - ToDoFunction 1.0 jar + ToDoFunction com.baeldung @@ -87,8 +87,27 @@ org.apache.maven.plugins maven-shade-plugin - 3.2.4 + ${maven-shade-plugin.version} + false + + + *:* + + **/module-info.class + META-INF/LICENSE + META-INF/LICENSE.md + META-INF/LICENSE.txt + META-INF/NOTICE + META-INF/NOTICE.md + META-INF/NOTICE.txt + META-INF/MANIFEST.MF + META-INF/DEPENDENCIES + META-INF/services/com.fasterxml.jackson.core.JsonFactory + META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat + + + @@ -105,6 +124,7 @@ 11 11 + 3.6.0 1.2.1 3.6.0 1.2.1 diff --git a/aws-modules/aws-miscellaneous/README.md b/aws-modules/aws-miscellaneous/README.md deleted file mode 100644 index 104c8719df1b..000000000000 --- a/aws-modules/aws-miscellaneous/README.md +++ /dev/null @@ -1,9 +0,0 @@ -## AWS Miscellaneous - -This module contains articles about various Amazon Web Services (AWS) such as EC2, DynamoDB, SQS, RDS - -### Relevant articles - -- [Managing EC2 Instances in Java](https://www.baeldung.com/ec2-java) -- [Managing Amazon SQS Queues in Java](https://www.baeldung.com/aws-queues-java) -- [Guide to AWS Aurora RDS with Java](https://www.baeldung.com/aws-aurora-rds-java) diff --git a/aws-modules/aws-miscellaneous/pom.xml b/aws-modules/aws-miscellaneous/pom.xml index 50d6a9c47402..29aac272e756 100644 --- a/aws-modules/aws-miscellaneous/pom.xml +++ b/aws-modules/aws-miscellaneous/pom.xml @@ -30,6 +30,18 @@ gson ${gson.version} + + org.mockito + mockito-core + 5.12.0 + test + + + org.mockito + mockito-junit-jupiter + 5.12.0 + test + diff --git a/aws-modules/aws-miscellaneous/src/main/java/com/baeldung/sqs/SqsAsyncMessagePublisher.java b/aws-modules/aws-miscellaneous/src/main/java/com/baeldung/sqs/SqsAsyncMessagePublisher.java new file mode 100644 index 000000000000..40b596b37759 --- /dev/null +++ b/aws-modules/aws-miscellaneous/src/main/java/com/baeldung/sqs/SqsAsyncMessagePublisher.java @@ -0,0 +1,26 @@ +package com.baeldung.sqs; + +import software.amazon.awssdk.services.sqs.SqsAsyncClient; +import software.amazon.awssdk.services.sqs.model.SendMessageRequest; +import software.amazon.awssdk.services.sqs.model.SendMessageResponse; + +import java.util.concurrent.CompletableFuture; + +public class SqsAsyncMessagePublisher { + + private final SqsAsyncClient sqsAsyncClient; + + public SqsAsyncMessagePublisher(SqsAsyncClient sqsAsyncClient) { + this.sqsAsyncClient = sqsAsyncClient; + } + + public CompletableFuture publishMessage(String queueUrl, String messageBody) { + SendMessageRequest request = SendMessageRequest.builder() + .queueUrl(queueUrl) + .messageBody(messageBody) + .build(); + + return sqsAsyncClient.sendMessage(request) + .thenApply(SendMessageResponse::messageId); + } +} diff --git a/aws-modules/aws-miscellaneous/src/main/java/com/baeldung/sqs/SqsMessagePublisher.java b/aws-modules/aws-miscellaneous/src/main/java/com/baeldung/sqs/SqsMessagePublisher.java new file mode 100644 index 000000000000..25db63a1ca82 --- /dev/null +++ b/aws-modules/aws-miscellaneous/src/main/java/com/baeldung/sqs/SqsMessagePublisher.java @@ -0,0 +1,24 @@ +package com.baeldung.sqs; + +import software.amazon.awssdk.services.sqs.SqsClient; +import software.amazon.awssdk.services.sqs.model.SendMessageRequest; +import software.amazon.awssdk.services.sqs.model.SendMessageResponse; + +public class SqsMessagePublisher { + + private final SqsClient sqsClient; + + public SqsMessagePublisher(SqsClient sqsClient) { + this.sqsClient = sqsClient; + } + + public String publishMessage(String queueUrl, String messageBody) { + SendMessageRequest request = SendMessageRequest.builder() + .queueUrl(queueUrl) + .messageBody(messageBody) + .build(); + + SendMessageResponse response = sqsClient.sendMessage(request); + return response.messageId(); + } +} diff --git a/aws-modules/aws-miscellaneous/src/test/java/com/baeldung/sqs/SqsAsyncMessagePublisherUnitTest.java b/aws-modules/aws-miscellaneous/src/test/java/com/baeldung/sqs/SqsAsyncMessagePublisherUnitTest.java new file mode 100644 index 000000000000..6ec70799d1c8 --- /dev/null +++ b/aws-modules/aws-miscellaneous/src/test/java/com/baeldung/sqs/SqsAsyncMessagePublisherUnitTest.java @@ -0,0 +1,53 @@ +package com.baeldung.sqs; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.ArgumentCaptor; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import software.amazon.awssdk.services.sqs.SqsAsyncClient; +import software.amazon.awssdk.services.sqs.model.SendMessageRequest; +import software.amazon.awssdk.services.sqs.model.SendMessageResponse; +import java.util.concurrent.CompletableFuture; +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class SqsAsyncMessagePublisherUnitTest { + + @Mock + private SqsAsyncClient sqsAsyncClient; + + @InjectMocks + private SqsAsyncMessagePublisher messagePublisher; + + @Test + void whenPublishMessage_thenMessageIsSentAsynchronously() throws Exception { + // Arrange + String queueUrl = "https://sqs.us-east-1.amazonaws.com/123456789012/MyAsyncQueue"; + String messageBody = "Hello, Async SQS!"; + String expectedMessageId = "test-async-message-id-456"; + + SendMessageResponse mockResponse = SendMessageResponse.builder() + .messageId(expectedMessageId) + .build(); + when(sqsAsyncClient.sendMessage(any(SendMessageRequest.class))) + .thenReturn(CompletableFuture.completedFuture(mockResponse)); + + // Act + String actualMessageId = messagePublisher.publishMessage(queueUrl, messageBody).get(); + + // Assert + assertThat(actualMessageId).isEqualTo(expectedMessageId); + + ArgumentCaptor requestCaptor = ArgumentCaptor.forClass(SendMessageRequest.class); + verify(sqsAsyncClient).sendMessage(requestCaptor.capture()); + + SendMessageRequest capturedRequest = requestCaptor.getValue(); + assertThat(capturedRequest.queueUrl()).isEqualTo(queueUrl); + assertThat(capturedRequest.messageBody()).isEqualTo(messageBody); + } +} \ No newline at end of file diff --git a/aws-modules/aws-miscellaneous/src/test/java/com/baeldung/sqs/SqsMessagePublisherUnitTest.java b/aws-modules/aws-miscellaneous/src/test/java/com/baeldung/sqs/SqsMessagePublisherUnitTest.java new file mode 100644 index 000000000000..c71d8638d463 --- /dev/null +++ b/aws-modules/aws-miscellaneous/src/test/java/com/baeldung/sqs/SqsMessagePublisherUnitTest.java @@ -0,0 +1,52 @@ +package com.baeldung.sqs; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.ArgumentCaptor; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import software.amazon.awssdk.services.sqs.SqsClient; +import software.amazon.awssdk.services.sqs.model.SendMessageRequest; +import software.amazon.awssdk.services.sqs.model.SendMessageResponse; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class SqsMessagePublisherUnitTest { + + @Mock + private SqsClient sqsClient; + + @InjectMocks + private SqsMessagePublisher messagePublisher; + + @Test + void whenPublishMessage_thenMessageIsSentWithCorrectParameters() { + // Arrange + String queueUrl = "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue"; + String messageBody = "Hello, SQS!"; + String expectedMessageId = "test-message-id-123"; + + SendMessageResponse mockResponse = SendMessageResponse.builder() + .messageId(expectedMessageId) + .build(); + when(sqsClient.sendMessage(any(SendMessageRequest.class))).thenReturn(mockResponse); + + // Act + String actualMessageId = messagePublisher.publishMessage(queueUrl, messageBody); + + // Assert + assertThat(actualMessageId).isEqualTo(expectedMessageId); + + ArgumentCaptor requestCaptor = ArgumentCaptor.forClass(SendMessageRequest.class); + verify(sqsClient).sendMessage(requestCaptor.capture()); + + SendMessageRequest capturedRequest = requestCaptor.getValue(); + assertThat(capturedRequest.queueUrl()).isEqualTo(queueUrl); + assertThat(capturedRequest.messageBody()).isEqualTo(messageBody); + } +} \ No newline at end of file diff --git a/aws-modules/aws-reactive/README.md b/aws-modules/aws-reactive/README.md deleted file mode 100644 index 9164bd0ea61a..000000000000 --- a/aws-modules/aws-reactive/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## AWS Reactive - -This module contains articles about reactive support with AWS - -### Relevant Articles: - -- [AWS S3 with Java – Reactive Support](https://www.baeldung.com/java-aws-s3-reactive) diff --git a/aws-modules/aws-rest/README.md b/aws-modules/aws-rest/README.md deleted file mode 100644 index 7961da54f192..000000000000 --- a/aws-modules/aws-rest/README.md +++ /dev/null @@ -1,10 +0,0 @@ -## AWS SpringBoot Rest - -This module contains articles about AWS access in Spring boot Rest APIs - -### Relevant Articles: -- [Download File from S3 Given a URL](https://www.baeldung.com/java-aws-download-file-s3-url) - - - - diff --git a/aws-modules/aws-rest/pom.xml b/aws-modules/aws-rest/pom.xml index a953267f4911..cd4a888d3b04 100644 --- a/aws-modules/aws-rest/pom.xml +++ b/aws-modules/aws-rest/pom.xml @@ -1,7 +1,7 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 com.baeldung aws-rest @@ -56,7 +56,6 @@ - org.springframework.boot spring-boot-devtools @@ -94,6 +93,15 @@ + + + default + + true + + + + 21 21 @@ -102,6 +110,8 @@ 1.18.32 5.5.2 2.16.1 + UTF-8 + UTF-8 \ No newline at end of file diff --git a/aws-modules/aws-s3-2/README.md b/aws-modules/aws-s3-2/README.md deleted file mode 100644 index e4c9d88c33e3..000000000000 --- a/aws-modules/aws-s3-2/README.md +++ /dev/null @@ -1,5 +0,0 @@ -## AWS S3 - -This module contains articles about Simple Storage Service (S3) on AWS - -### Relevant articles diff --git a/aws-modules/aws-s3-2/pom.xml b/aws-modules/aws-s3-2/pom.xml index ea3682f089a6..8481a549a206 100644 --- a/aws-modules/aws-s3-2/pom.xml +++ b/aws-modules/aws-s3-2/pom.xml @@ -1,5 +1,7 @@ - + 4.0.0 aws-s3-2 0.1.0-SNAPSHOT diff --git a/aws-modules/aws-s3/README.md b/aws-modules/aws-s3/README.md deleted file mode 100644 index 55daa321c981..000000000000 --- a/aws-modules/aws-s3/README.md +++ /dev/null @@ -1,16 +0,0 @@ -## AWS S3 - -This module contains articles about Simple Storage Service (S3) on AWS - -### Relevant articles - -- [Amazon S3 With Java](https://www.baeldung.com/java-aws-s3) -- [Multipart Uploads in Amazon S3 with Java](https://www.baeldung.com/aws-s3-multipart-upload) -- [Using the JetS3t Java Client With Amazon S3](https://www.baeldung.com/jets3t-amazon-s3) -- [Check if a Specified Key Exists in a Given S3 Bucket Using Java](https://www.baeldung.com/java-aws-s3-check-specified-key-exists) -- [Listing All AWS S3 Objects in a Bucket Using Java](https://www.baeldung.com/java-aws-s3-list-bucket-objects) -- [Update an Existing Amazon S3 Object Using Java](https://www.baeldung.com/java-update-amazon-s3-object) -- [How To Rename Files and Folders in Amazon S3](https://www.baeldung.com/java-amazon-s3-rename-files-folders) -- [Update an Existing Amazon S3 Object Using Java](https://www.baeldung.com/java-update-amazon-s3-object) -- [How to Mock Amazon S3 for Integration Test](https://www.baeldung.com/java-amazon-simple-storage-service-mock-testing) -- More articles: [[next -->]](../aws-s3-2) diff --git a/aws-modules/aws-s3/pom.xml b/aws-modules/aws-s3/pom.xml index 4b6e88b087d0..a4dce77b18cc 100644 --- a/aws-modules/aws-s3/pom.xml +++ b/aws-modules/aws-s3/pom.xml @@ -30,7 +30,6 @@ commons-codec ${commons-codec.version} - software.amazon.awssdk diff --git a/aws-modules/pom.xml b/aws-modules/pom.xml index d6348af25fb2..f15b466944f2 100644 --- a/aws-modules/pom.xml +++ b/aws-modules/pom.xml @@ -18,6 +18,7 @@ amazon-textract aws-app-sync aws-dynamodb + aws-dynamodb-v2 aws-lambda-modules aws-miscellaneous aws-reactive @@ -27,20 +28,46 @@ s3proxy + + + + software.amazon.awssdk + bom + ${aws-java-sdk-v2.version} + pom + import + + + com.amazonaws + aws-java-sdk-bom + ${aws-java-sdk.version} + pom + import + + + + com.amazonaws aws-java-sdk-dynamodb ${aws-java-sdk-dynamodb.version} compile + + + commons-logging + commons-logging + + - 1.12.331 + 1.12.777 2.24.9 3.0.0 1.12.523 + 3.3.2 diff --git a/aws-modules/s3proxy/pom.xml b/aws-modules/s3proxy/pom.xml index 6e5ccdbe1c6c..a8b4a83fecbd 100644 --- a/aws-modules/s3proxy/pom.xml +++ b/aws-modules/s3proxy/pom.xml @@ -11,16 +11,31 @@ com.baeldung - parent-boot-3 - 0.0.1-SNAPSHOT - ../../parent-boot-3 + aws-modules + 1.0.0-SNAPSHOT + + + + org.springframework.boot + spring-boot-dependencies + ${spring-boot.version} + pom + import + + + + org.springframework.boot spring-boot-starter-web + + org.springframework.boot + spring-boot-starter-test + org.springframework.boot spring-boot-configuration-processor @@ -33,13 +48,11 @@ software.amazon.awssdk s3 - ${aws-sdk.version} 2.3.0 - 2.28.23 \ No newline at end of file diff --git a/azure-functions/README.md b/azure-functions/README.md deleted file mode 100644 index 1884b2abab32..000000000000 --- a/azure-functions/README.md +++ /dev/null @@ -1,2 +0,0 @@ -## Relevant Article -- [Azure Functions in Java](https://www.baeldung.com/java-azure-functions) diff --git a/azure-functions/pom.xml b/azure-functions/pom.xml index 5c3c0d3e7eb0..610a9e12daef 100644 --- a/azure-functions/pom.xml +++ b/azure-functions/pom.xml @@ -1,10 +1,12 @@ - + 4.0.0 azure-functions 1.0.0-SNAPSHOT - azure-functions jar + azure-functions com.baeldung @@ -12,20 +14,12 @@ 1.0.0-SNAPSHOT - - UTF-8 - 1.24.0 - 3.0.0 - azure-functions-1722835137455 - - com.microsoft.azure.functions azure-functions-java-library ${azure.functions.java.library.version} - @@ -33,7 +27,7 @@ org.apache.maven.plugins maven-compiler-plugin - 3.8.1 + ${maven-compiler-plugin.version} ${java.version} ${java.version} @@ -98,7 +92,7 @@ maven-clean-plugin - 3.1.0 + ${maven-clean-plugin.version} @@ -109,4 +103,13 @@ + + + UTF-8 + 1.24.0 + 3.0.0 + azure-functions-1722835137455 + 3.1.0 + + diff --git a/azure/README.md b/azure/README.md deleted file mode 100644 index 4da8481502ea..000000000000 --- a/azure/README.md +++ /dev/null @@ -1,8 +0,0 @@ -## Azure - -This module contains articles about Azure - -### Relevant Articles: - -- [Deploy a Spring Boot App to Azure](https://www.baeldung.com/spring-boot-azure) - diff --git a/azure/pom.xml b/azure/pom.xml deleted file mode 100644 index 09bc545f3373..000000000000 --- a/azure/pom.xml +++ /dev/null @@ -1,136 +0,0 @@ - - - 4.0.0 - azure - 0.1 - war - azure - Demo project for Spring Boot on Azure - - - com.baeldung - parent-boot-3 - 0.0.1-SNAPSHOT - ../parent-boot-3 - - - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-tomcat - provided - - - org.springframework.boot - spring-boot-starter-data-jpa - - - com.h2database - h2 - runtime - - - com.mysql - mysql-connector-j - runtime - - - - - - - org.springframework.boot - spring-boot-maven-plugin - - - com.spotify - docker-maven-plugin - {docker-maven-plugin.version} - - ${docker.image.prefix}/${project.artifactId} - https://${docker.image.prefix} - ${azure.containerRegistry} - docker - - - / - ${project.build.directory} - ${project.build.finalName}.jar - - - - - - com.microsoft.azure - azure-webapp-maven-plugin - ${azure-webapp-maven-plugin.version} - - - azure-auth - - baeldung-group - baeldung-webapp - baeldung-plan - {java.version} - - - - - - - - - - spring.datasource.url - jdbc:h2:file:~/test - - - - spring.datasource.username - sa - - - - spring.datasource.password - test - - - - - - - - - - - - - - - - - - javax.xml.bind - jaxb-api - ${jaxb-api.version} - - - - - - - - baeldungadr - ${azure.containerRegistry}.azurecr.io - 1.1.0 - 1.1.0 - 2.4.0-b180830.0359 - - - \ No newline at end of file diff --git a/bazel/README.md b/bazel/README.md deleted file mode 100644 index af4f901f4988..000000000000 --- a/bazel/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Bazel - -This module contains articles about Bazel - -### Relevant Articles: - -- [Building Java Applications with Bazel](https://www.baeldung.com/bazel-build-tool) diff --git a/bazel/pom.xml b/bazel/pom.xml index 52c59d3d1d56..bbeb2f8ca980 100644 --- a/bazel/pom.xml +++ b/bazel/pom.xml @@ -4,8 +4,8 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 bazel - bazel pom + bazel com.baeldung diff --git a/checker-framework/README.md b/checker-framework/README.md deleted file mode 100644 index 59dc2878a28b..000000000000 --- a/checker-framework/README.md +++ /dev/null @@ -1,7 +0,0 @@ -## Checker Plugin - -This module contains articles about the Checker Plugin - -### Relevant articles - -- [The Checker Framework – Pluggable Type Systems for Java](https://www.baeldung.com/checker-framework) diff --git a/choco-solver/pom.xml b/choco-solver/pom.xml index 416e516630dc..02738dfecc45 100644 --- a/choco-solver/pom.xml +++ b/choco-solver/pom.xml @@ -1,21 +1,22 @@ + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 - choco-solver 1.0-SNAPSHOT + com.baeldung parent-modules 1.0.0-SNAPSHOT + org.choco-solver choco-solver - 4.10.14 + ${choco-solver.version} org.slf4j @@ -25,10 +26,8 @@ - 21 - 21 - UTF-8 1.7.21 + 4.10.14 \ No newline at end of file diff --git a/clojure-modules/clojure-ring/README.md b/clojure-modules/clojure-ring/README.md index 20263c6b95d4..3bc04ac12993 100644 --- a/clojure-modules/clojure-ring/README.md +++ b/clojure-modules/clojure-ring/README.md @@ -14,6 +14,3 @@ Firstly, start the REPL with `lein repl`. Then the examples can be executed with * `(run request-count-handler)` - A handler with a session that tracks how many times this session has requested this handler In all cases, the handlers can be accessed on http://localhost:3000. - -## Relevant Articles -- [Writing Clojure Webapps with Ring](https://www.baeldung.com/clojure-ring) diff --git a/codenameone/.agent-skills/codename-one/SKILL.md b/codenameone/.agent-skills/codename-one/SKILL.md new file mode 100644 index 000000000000..d6f730b088a8 --- /dev/null +++ b/codenameone/.agent-skills/codename-one/SKILL.md @@ -0,0 +1,297 @@ +--- +name: codename-one +description: Build and modify Codename One cross-platform mobile apps (Java 17, Maven, ParparVM/Android/iOS/JavaScript). Use when the project contains a `common/codenameone_settings.properties`, depends on `com.codenameone:codenameone-core`, edits CSS files under `common/src/main/css/`, calls `cn1:run`, `cn1:test`, `cn1:build`, references `com.codename1.ui.*` / `com.codename1.testing.*`, or when the user asks to build a UI, write screen tests, generate screenshots, or compare to Swing/HTML. +metadata: + type: skill +--- + +# Codename One — App and UI Authoring Skill + +This skill teaches you how to write code for a Codename One (CN1) cross-platform mobile project. Codename One compiles Java/Kotlin bytecode to native iOS, Android, desktop and web. It looks like Java AWT/Swing, behaves like a mobile UI toolkit, and styles with a subset of CSS. + +**Use this skill when**: + +- A file you are editing imports `com.codename1.ui.*`, `com.codename1.io.*`, `com.codename1.testing.*`, or extends `com.codename1.system.Lifecycle`. +- You are editing a file in `common/src/main/css/` (CN1 CSS). +- You are running `cn1:run`, `cn1:debug`, `cn1:test`, or `cn1:build` Maven goals. +- The user asks for a UI screen, a screenshot test, a responsive layout, or wants to convert a Swing/HTML snippet to CN1. + +## How this skill is organized + +`SKILL.md` (this file) is the top-level cheat sheet. Deeper reference material lives under `references/` — pull the relevant file in **only when you need it**: + +- `references/build-and-run.md` — Local vs cloud builds, JDK matrix, Maven goals, `codenameone_settings.properties`, running the simulator, building for iOS/Android/Web, automated (Enterprise) cloud builds in CI. +- `references/build-hints.md` — Curated index of `codename1.arg.*` build hints (iOS, Android, push, web). +- `references/java-api-subset.md` — How to inspect the supported Java API subset, IO (`Storage`, `FileSystemStorage`), networking (`ConnectionRequest`, `Rest`), concurrency, dates, SQLite. **Read this whenever the compliance check fails or when you reach for a `java.*` API.** +- `references/ui-components.md` — Form, Toolbar, Container layouts (Border/Box/Flow/Grid/Layered), common components, navigation, dialogs. +- `references/css.md` — CSS capabilities and (important) **limitations**. Selectors, supported properties, 9-patch borders, theme constants. +- `references/swing-comparison.md` — Mapping Swing concepts and code to Codename One. Read this when porting Swing code. +- `references/html-css-cheatsheet.md` — Converting common HTML/CSS snippets to CN1 components + CSS. +- `references/android-to-cn1.md` — Porting Android (XML + Kotlin/Java) screens to Codename One. +- `references/testing-and-screenshots.md` — `AbstractTest`, `TestUtils`, `screenshotTest`, the `cn1:test` Maven goal, the screenshot tolerance algorithm. +- `references/junit-testing.md` — Standard JUnit 5 tests against the simulator via `@CodenameOneTest`. Annotations (`@RunOnEdt`, `@Theme`, `@DarkMode`, `@LargerText`, `@Orientation`, `@RTL`, `@SimulatorProperty`), how it coexists with `cn1:test`, and why a headless CI runner has to be configured with Xvfb (or accepts that JUnit test classes will be skipped). +- `references/mobile-adaptability.md` — Density-independent units (mm), `convertToPixels`, `LayeredLayout` for responsive design, `Display.isTablet()`, font scaling. +- `references/native-interfaces.md` — Authoring native interfaces for iOS/Android/JavaScript/Desktop with `cn1:generate-native-interfaces` and platform callbacks. +- `references/cn1libs.md` — Creating, packaging, and consuming Codename One libraries (Maven and legacy `.cn1lib`). +- `references/snapshot-builds.md` — Edge case: compiling against a Codename One SNAPSHOT from git. +- `references/debugging.md` — `jdb`-attach workflow for an agent: start the simulator paused, set breakpoints, dump locals, drive the session non-interactively from a script. +- `tools/` — runnable Java 17 single-file utilities. `tools/IsApiSupported.java` answers "is this `java.*` class in the CN1 subset?"; `tools/IsCssValid.java` answers "does this `theme.css` compile?". Run with `java tools/.java `. + +When the user's task hits any one of those topics, **read the matching reference before generating code**. Do not paste large snippets without checking. + +## Project layout (multi-module Maven) + +A CN1 project generated by the initializr has these modules: + +``` +my-app/ +├── pom.xml # Aggregator. cn1.plugin.version + cn1.version pinned here. +├── common/ # Cross-platform Java/Kotlin source. THIS IS WHERE THE APP LIVES. +│ ├── pom.xml # 17 17 by default +│ ├── codenameone_settings.properties +│ └── src/main/ +│ ├── java//.java +│ ├── css/theme.css # CN1 CSS (NOT regular web CSS - see references/css.md) +│ ├── l10n/ # i18n bundles (NOT src/main/resources!) +│ └── guibuilder/ # Optional GUI builder XML +├── javase/ # Desktop simulator port +├── android/ # Android wrapper (built via build server or local Gradle) +├── ios/ # iOS wrapper (ParparVM) +└── javascript/ # TeaVM-based web port +``` + +**Only edit `common/`**. The platform modules are thin wrappers — touching them is almost always wrong unless you are intentionally writing a native interface. + +## Java version and language features + +This project targets **Java 17** (`17` / `17` in `common/pom.xml`, plus `codename1.arg.java.version=17` in `codenameone_settings.properties`). Use: + +- `var` for local variable type inference +- Text blocks (`"""..."""`) +- Records +- Pattern matching for `instanceof` +- `switch` expressions +- Lambdas, method references, `Stream`s + +**Caveat — the build server cross-compiles to bytecode that ParparVM/TeaVM can consume.** Codename One ships a curated subset of the JDK, **not** the full `java.*` namespace. The `cn1:compliance-check` Maven goal runs on every compile and fails the build if you call an unsupported API. The most common gotchas: + +- No `java.nio.file.*` — use `com.codename1.io.FileSystemStorage` and `Storage`. +- No `java.net.http.*` / `java.net.URLConnection` — use `com.codename1.io.rest.Rest` (preferred) or `ConnectionRequest`. +- No `java.util.concurrent.locks.*` beyond simple `synchronized` — use `Display.getInstance().callSerially(...)` or `Display.startThread(...)`. +- No `java.awt.*` / `javax.swing.*` — CN1 has its own UI stack. See `references/swing-comparison.md`. +- No `java.lang.reflect.*` on production builds — works in the simulator only. +- No threads spawned with `new Thread(...).start()` for UI work — always go through `Display.callSerially` or `Display.startThread(...)`. + +For the authoritative subset list and IO/networking patterns, read `references/java-api-subset.md` (which also shows how to grep the `java-runtime` jar to verify any specific class/method). + +## The Event Dispatch Thread (EDT) + +CN1 has a single EDT, exactly like Swing. All UI mutation **must** happen on it. + +- Inside event listeners and lifecycle callbacks (`start`, `stop`, `init`) you are already on the EDT. +- From a background thread, hop back with `Display.getInstance().callSerially(() -> { ... })` (or `callSeriallyAndWait` if you need to block). +- Use `Display.getInstance().startThread(runnable, "name").start()` instead of `new Thread(...)` so cleanup happens correctly across platforms. + +`references/swing-comparison.md` contains a Swing→CN1 EDT idiom table. + +## The Lifecycle main class + +Every CN1 app extends `com.codename1.system.Lifecycle` (or `com.codename1.ui.util.Lifecycle` in older code). The four methods you may override: + +```java +public class MyAppName extends Lifecycle { + @Override + public void init(Object context) { + // Called once on the EDT. The Lifecycle base class already installs + // the theme; reach for the cached global resources instance from + // here on (Resources.getGlobalResources() returns the in-RAM copy, + // no disk re-read). + } + + @Override + public void runApp() { + // Build and show the first form. + Form f = new Form("Hello", new BorderLayout()); + f.add(BorderLayout.CENTER, new Label("Welcome")); + f.show(); + } + + @Override + public void stop() { /* App backgrounded */ } + + @Override + public void destroy() { /* App killed */ } +} +``` + +## Minimal "first screen" pattern + +```java +import static com.codename1.ui.CN.*; // Convenience statics: callSerially, etc. +import com.codename1.ui.*; +import com.codename1.ui.layouts.*; + +Form f = new Form("Profile", BoxLayout.y()); +f.getToolbar().addCommandToRightBar("Save", null, e -> save()); +f.add(new Label("Name")) + .add(new TextField()) + .add(new Button("Submit")); +f.show(); +``` + +`BoxLayout.y()` (vertical) and `BoxLayout.x()` (horizontal) are the most common layouts. Wrap a `Form` content pane in `BorderLayout` when you want a header/footer/center split. See `references/ui-components.md` for the full layout matrix. + +## CSS in Codename One + +CN1 ships with a CSS compiler that bakes `common/src/main/css/theme.css` into the binary theme resource (`theme.res`). It supports a deliberate **subset** of web CSS: + +```css +Form { + background-color: #0f172a; /* hex, rgb(), or named colors */ + padding: 2mm; /* mm is the recommended unit */ +} + +Button { + background-color: #1d4ed8; + color: #ffffff; + border: 1px solid #1d4ed8; + border-radius: 3mm; + padding: 2mm 4mm; +} + +Button.pressed { /* state pseudo-class baked as UIID */ + background-color: #1e3a8a; +} + +#Constants { + useLargerTextScaleBool: true; /* theme constants, not standard CSS */ +} +``` + +**Key differences from web CSS** (read `references/css.md` before authoring more): + +- Selectors target **UIIDs** (Codename One component style names), not arbitrary HTML elements. `Button`, `Form`, `Label`, `Toolbar`, `Title` are the most common. +- No descendant combinator, no `:hover`, no media queries. State variants are baked: `.pressed`, `.disabled`, `.selected`. +- Units: prefer `mm` (millimeters) over `px`. CN1 converts `mm` to device pixels via `Display.convertToPixels`. `1mm` ≈ 6-9 px depending on density. +- `border-radius` works but is rasterized at compile time — animating it at runtime requires programmatic styling. +- No `transform`, no `flex`, no `grid`. Use CN1 Java layouts for arrangement; CSS is only for *styling*. +- Bundled named colors are limited: `pink`, `orange`, `purple`, `yellow`, `gray`/`grey` are translated to hex by the initializr, anything else you must specify as hex. + +`references/html-css-cheatsheet.md` shows how to map "I want a flexbox row" / "I want a hero section" / "I want a card" to CN1 idioms. + +## Adaptability and responsive design + +Mobile screens vary wildly. CN1 gives you: + +- **Density-independent units**: `1mm` always renders ~1mm tall regardless of pixel density. Always size in `mm`, not `px`. +- `Display.getInstance().convertToPixels(2.5f)` — convert millimeters to current device pixels programmatically. +- `Display.getInstance().isTablet()`, `Display.getInstance().isPortrait()`, `Display.getInstance().getDisplayWidth/Height()` — branch on form factor. +- `LayeredLayout` with `LayeredLayoutConstraint` for precise responsive positioning (percent-based insets). +- `Toolbar` automatically reshapes to platform conventions (Android side menu / iOS tab bar). + +See `references/mobile-adaptability.md` for patterns: phone-vs-tablet master-detail, orientation listeners, dynamic font scaling. + +## Testing + +CN1 supports two compatible test styles in the same project: + +1. **Legacy `AbstractTest` + `cn1:test`.** Required for tests that must also run on a device (`mvn cn1:test -Dtarget=ios`). Compiles under the device subset (no reflection, no JavaSE APIs). See `references/testing-and-screenshots.md`. +2. **Standard JUnit 5 + `@CodenameOneTest`.** Runs only in the simulator JVM via Surefire, so you get reflection, Mockito, AssertJ, IDE green-bar integration, `-Dtest=Foo#bar` filtering. Faster startup. See `references/junit-testing.md`. + +Both runners coexist — `cn1:test` discovers `UnitTest` implementers, Surefire discovers `@Test` methods, they don't trip over each other. Pick per test class. + +```java +// Legacy AbstractTest -- compiles under the device subset, runs via `cn1:test`. +public class LoginFormTest extends AbstractTest { + @Override public boolean shouldExecuteOnEDT() { return true; } + @Override public boolean runTest() throws Exception { + new MyAppName().runApp(); + TestUtils.waitForFormTitle("Login"); + TestUtils.setText("usernameField", "alice"); + TestUtils.clickButtonByLabel("Sign In"); + TestUtils.waitForFormTitle("Home"); + return screenshotTest("home-screen-baseline"); + } +} + +// JUnit 5 -- simulator-only, runs via `mvn test` / Surefire. +@CodenameOneTest +class GreetingFormTest { + @Test + @RunOnEdt + void formShowsExpectedTitle() { + new Form("Hello").show(); + assertEquals("Hello", Display.getInstance().getCurrent().getTitle()); + } +} +``` + +Run with `mvn -pl common cn1:test` (cn1:test runner only) or `mvn test` (both runners). The cn1app archetype already wires up Surefire + JUnit Jupiter in the generated POMs. + +`screenshotTest(name)` captures the current form, compares against a stored baseline under `Storage`, and returns `true` if within tolerance. First run records the baseline. See `references/testing-and-screenshots.md` for the tolerance algorithm and how to validate UI you just wrote. + +> Important: a "screenshot matches baseline" only proves consistency, **not** correctness. If you just generated the baseline yourself, you have not validated the screen — visually inspect at least once before treating that baseline as ground truth. + +> Headless caveat: any simulator-driven test (both flavors) needs an X server / Xvfb to construct the simulator's `JFrame`. The `@CodenameOneTest` extension auto-aborts the class on a headless JVM so you get "skipped" instead of "errored"; the `cn1:test` runner needs you to skip with `-DskipTests` or run under `xvfb-run`. + +## Build and run commands + +From the project root: + +```bash +# Run in the desktop simulator (requires JDK 11–25 at runtime; build still uses JDK 17 source level) +mvn -pl common cn1:run + +# Run with breakpoints +mvn -pl common cn1:debug + +# Execute the CN1 test runner +mvn -pl common cn1:test + +# Cloud build for Android/iOS/JS (requires CN1 build server creds) +mvn -pl android package -Dcodename1.platform=android -Dcodename1.buildTarget=android-device +mvn -pl ios package -Dcodename1.platform=ios -Dcodename1.buildTarget=ios-device +mvn -pl javascript package -Dcodename1.platform=javascript -Dcodename1.buildTarget=javascript +``` + +See `references/build-and-run.md` for the local-vs-cloud matrix, automated-build mode (Enterprise), iOS local-build prerequisites, and the complete goal list. The full `codename1.arg.*` index lives in `references/build-hints.md`. + +## What NOT to do + +- Don't use `java.awt.Color` / `java.awt.Font` / `javax.swing.*` — CN1 has its own `Color` constants (just `int` ARGB), `Font.createTrueTypeFont`, and `Component` hierarchy. +- Don't add CSS that references web-only properties (`display`, `flex`, `position`, `transform`, `@media`) — the CN1 CSS compiler will silently ignore them or fail. +- Don't put localization bundles under `common/src/main/resources/`. The CN1 plugin scans `common/src/main/l10n/` (or `common/src/main/i18n/`); bundles placed anywhere else are NOT baked into `theme.res` and `Resources.getL10N("messages", lang)` returns `null` at runtime. +- Don't spin up `new Thread(...)` for UI work — use `Display.getInstance().callSerially(...)` or `Display.startThread(...)`. +- Don't mutate UI off the EDT. Symptoms: random repaint glitches, native crashes on iOS. +- Don't write screenshot tests where the baseline was just generated by the same code you are validating — that proves nothing. + +## Sanity-check loop before reporting "done" + +For any UI-altering change: + +1. Run `mvn -pl common cn1:run` in the simulator and click through the changed flow. +2. Inspect at least one screenshot (capture via the simulator menu → Save Screenshot, or generate via a test). +3. Resize the simulator window or toggle a different skin to confirm the layout doesn't break on a different form factor. +4. If you wrote a `screenshotTest`, delete the auto-generated baseline once if the screen has changed, then re-run twice — the second run should pass with `true`. + +If you cannot run the simulator (e.g. headless environment), **say so explicitly in the response** rather than claiming the UI works. + +## Reference quick-look index + +| If the user asks for... | Open this reference | +| --- | --- | +| "Add a screen with a list / form / dialog" | `references/ui-components.md` | +| "Make this look like X" / CSS tweaks | `references/css.md` | +| "Port this from Swing" / Swing idioms | `references/swing-comparison.md` | +| "I have HTML/CSS, convert it" | `references/html-css-cheatsheet.md` | +| "I have Android XML/Kotlin/Java, convert it" | `references/android-to-cn1.md` | +| "Write a test for this screen" / "Compare to a baseline" | `references/testing-and-screenshots.md` | +| "Make it look right on tablet/landscape" | `references/mobile-adaptability.md` | +| "How do I run/build/deploy" | `references/build-and-run.md` | +| "What's the right `codename1.arg.*` for X" / native config | `references/build-hints.md` | +| "Why does the compliance check fail" / Java/IO/networking | `references/java-api-subset.md` | +| "I need to call a native iOS/Android/JS/desktop API" | `references/native-interfaces.md` | +| "How do I create / consume a cn1lib" | `references/cn1libs.md` | +| "Build against a Codename One SNAPSHOT from git" | `references/snapshot-builds.md` | +| "Debug a faulty screen — attach `jdb` to the simulator" | `references/debugging.md` | +| Quick yes/no check: "is this `java.*` class supported", "does my `theme.css` compile" | `tools/` directory — `java tools/IsApiSupported.java ` / `java tools/IsCssValid.java ` | diff --git a/codenameone/.agent-skills/codename-one/references/android-to-cn1.md b/codenameone/.agent-skills/codename-one/references/android-to-cn1.md new file mode 100644 index 000000000000..96654d3b9ede --- /dev/null +++ b/codenameone/.agent-skills/codename-one/references/android-to-cn1.md @@ -0,0 +1,140 @@ +# Porting Android (XML + Kotlin/Java) to Codename One + +Codename One's component model is similar enough to Android's that you can usually translate screens one-to-one. As with the HTML conversion guide (`references/html-css-cheatsheet.md`), the layout system is the part that differs most — Android XML doesn't translate; build the same screen in Java + CN1 CSS. + +## Android view → CN1 component + +| Android `View` | CN1 component | Notes | +| --- | --- | --- | +| `LinearLayout` (vertical) | `BoxLayout.y()` | | +| `LinearLayout` (horizontal) | `BoxLayout.x()` | | +| `RelativeLayout` / `ConstraintLayout` | `LayeredLayout` with `LayeredLayoutConstraint` | Percent or mm insets and `setReferenceComponent*` for "below this view". | +| `FrameLayout` | `LayeredLayout` | Same stacking semantics. | +| `GridLayout` (Android's, not CSS) | `GridLayout` | | +| `RecyclerView` | `InfiniteContainer` | The pagination/recycling story maps directly. | +| `ScrollView` (vertical) | `Container.setScrollableY(true)` | Wrap your content. | +| `HorizontalScrollView` | `Container.setScrollableX(true)` | | +| `NestedScrollView` | **Don't nest scrollables in CN1** — see *Scrolling* in `references/ui-components.md`. | +| `Toolbar` / `ActionBar` | `Form.getToolbar()` | Already on every Form. Use `addMaterialCommandToRightBar/SideMenu` for actions. | +| `BottomNavigationView` | `Tabs` (placed in `BorderLayout.SOUTH`) or `Toolbar` bottom commands. | | +| `NavigationView` (drawer) | `Toolbar.addCommandToSideMenu(...)` / `addMaterialCommandToSideMenu(...)`. | | +| `TextView` | `Label` (single line) / `SpanLabel` (wrapped) | | +| `EditText` | `TextField` (single line) / `TextArea` (multi-line) | Set `setConstraint(...)` for the keyboard type. | +| `Button` / `MaterialButton` | `Button` | Use `Button(String, char)` to set a Material icon directly. | +| `ImageView` | `Label` with an `Image`, or `URLImage` for remote, or `Label.setMaterialIcon(char)` for an icon glyph. | | +| `Switch` / `CheckBox` / `RadioButton` | `Switch` / `CheckBox` / `RadioButton` | `RadioButton` needs a `ButtonGroup`. | +| `Spinner` | `Picker` (string list) | Do **not** use `ComboBox`. | +| `ProgressBar` (determinate) | `Slider` (set max, `setEditable(false)`) | | +| `ProgressBar` (indeterminate) | `InfiniteProgress` | | +| `FloatingActionButton` | `com.codename1.components.FloatingActionButton` | `createFAB(char icon).bindFabToContainer(...)`. | +| `Dialog` / `AlertDialog` | `Dialog.show(...)` | | +| `Toast` | `ToastBar.showMessage(...)` | | +| `Fragment` | A factory method returning a configured `Container`, attached to a Form. CN1 has no Fragment lifecycle — keep state in regular Java objects. | +| `Activity` | A separate `Form` class (or factory). Navigation = `nextForm.show()`. | +| `Intent` (in-app) | Direct method call to the next Form's factory. | +| `Intent` (external — `tel:` / `mailto:` / URL) | `Display.getInstance().execute("tel:...")`. | +| `RecyclerView.Adapter` | Implement `InfiniteContainer.fetchComponents(int, int)` or pass items to `MultiList`. | +| `WebView` | `BrowserComponent`. | +| `BottomSheetDialogFragment` | `InteractionDialog` (slides up from the bottom of the layered pane). | + +## Android XML → CN1 Java + +CN1 has no XML layout for screens (the GUI builder uses its own format). Translate Android XML directly to Java: + +```xml + + + + +