Add Spark CBO config tips for boosting motif finding performance (#845)

WeichenXu123 · Copilot · web-flow · commit 0d634b648f27 · 2026-06-08T17:06:24.000+02:00
* init

Signed-off-by: Weichen Xu &lt;weichen.xu@databricks.com&gt;

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI &lt;175728472+Copilot@users.noreply.github.com&gt;

* update

Signed-off-by: Weichen Xu &lt;weichen.xu@databricks.com&gt;

* format

Signed-off-by: Weichen Xu &lt;weichen.xu@databricks.com&gt;

---------

Signed-off-by: Weichen Xu &lt;weichen.xu@databricks.com&gt;
Co-authored-by: Copilot Autofix powered by AI &lt;175728472+Copilot@users.noreply.github.com&gt;
diff --git a/core/src/main/scala/org/graphframes/GraphFrame.scala b/core/src/main/scala/org/graphframes/GraphFrame.scala
@@ -592,6 +592,23 @@ class GraphFrame private (
    * This can return duplicate rows. E.g., a query `"(u)-[]->()"` will return a result for each
    * matching edge, even if those edges share the same vertex `u`.
    *
+   * ==Performance==
+   * Motif finding translates patterns into a series of joins. Enabling Spark's Cost-Based
+   * Optimizer (CBO) and join reordering can significantly improve performance by letting Spark
+   * choose more efficient join orderings based on table statistics:
+   * {{{
+   * spark.conf.set("spark.sql.cbo.enabled", "true")
+   * spark.conf.set("spark.sql.cbo.joinReorder.enabled", "true")
+   * }}}
+   * The join reorder algorithm is bounded by `spark.sql.cbo.joinReorder.dp.threshold` (default:
+   * `12`). If the estimated number of joins in your motif exceeds this threshold, increase it
+   * accordingly:
+   * {{{
+   * spark.conf.set("spark.sql.cbo.joinReorder.dp.threshold", "20")
+   * }}}
+   * CBO relies on table statistics, so run `ANALYZE TABLE <tableName> COMPUTE STATISTICS` on the
+   * vertices and edges tables to ensure accurate statistics are available.
+   *
    * @param pattern
    *   Pattern specifying a motif to search for.
    * @return
diff --git a/docs/src/04-user-guide/04-motif-finding.md b/docs/src/04-user-guide/04-motif-finding.md
@@ -55,6 +55,23 @@ can be expressed by applying filters to the result `DataFrame`.
 This can return duplicate rows.  E.g., a query `"(u)-[]->()"` will return a result for each
 matching edge, even if those edges share the same vertex `u`.
 
+## Performance
+
+Motif finding translates structural patterns into a series of joins. Enabling Spark's Cost-Based Optimizer (CBO) and join reordering allows Spark to pick more efficient join orderings based on table statistics, which can significantly boost motif finding performance:
+
+```
+spark.conf.set("spark.sql.cbo.enabled", "true")
+spark.conf.set("spark.sql.cbo.joinReorder.enabled", "true")
+```
+
+The join reorder algorithm uses dynamic programming and is bounded by `spark.sql.cbo.joinReorder.dp.threshold` (default: `12`). If the estimated number of joins in your motif exceeds this threshold, increase it accordingly:
+
+```
+spark.conf.set("spark.sql.cbo.joinReorder.dp.threshold", "20")
+```
+
+Note that CBO relies on table statistics. Run `ANALYZE TABLE <tableName> COMPUTE STATISTICS` on the vertices and edges tables, or use `spark.sql("ANALYZE TABLE ...")`, to ensure accurate statistics are available.
+
 ## Python API
 
 For API details, refer to the @:pydoc(graphframes.GraphFrame.find).
diff --git a/python/graphframes/graphframe.py b/python/graphframes/graphframe.py
@@ -401,12 +401,32 @@ def pregel(self) -> Pregel:
 
     def find(self, pattern: str) -> DataFrame:
         """
-        Motif finding.
+        Motif finding: searching the graph for structural patterns.
 
-        See Scala documentation for more details.
+        Motif finding uses a simple Domain-Specific Language (DSL) for expressing structural
+        queries. For example, ``graph.find("(a)-[e1]->(b); (b)-[e2]->(a)")`` will search for
+        pairs of vertices ``a``, ``b`` connected by edges in both directions. It returns a
+        :class:`DataFrame` of all such structures, with columns for each named element (vertex
+        or edge) in the motif.
+
+        **Performance tip:** Motif finding translates patterns into a series of joins. Enabling
+        Spark's Cost-Based Optimizer (CBO) and join reordering can significantly improve
+        performance::
+
+            spark.conf.set("spark.sql.cbo.enabled", "true")
+            spark.conf.set("spark.sql.cbo.joinReorder.enabled", "true")
+
+        The join reorder algorithm is bounded by ``spark.sql.cbo.joinReorder.dp.threshold``
+        (default: ``12``). If the estimated number of joins in your motif exceeds this threshold,
+        increase it accordingly::
+
+            spark.conf.set("spark.sql.cbo.joinReorder.dp.threshold", "20")
+
+        CBO relies on table statistics, so run ``ANALYZE TABLE <tableName> COMPUTE STATISTICS`` on
+        the vertices and edges tables (or temp views) to ensure accurate statistics are available.
 
-        :param pattern:  String describing the motif to search for.
-        :return:  DataFrame with one Row for each instance of the motif found
+        :param pattern: String describing the motif to search for.
+        :return: DataFrame with one Row for each instance of the motif found.
         """
         return self._impl.find(pattern=pattern)
 
