Skip to content

Commit 67f940e

Browse files
steve-ssteve-s
committed
Improvements in build tools docs
1 parent 72c627a commit 67f940e

1 file changed

Lines changed: 46 additions & 37 deletions

File tree

‎docs/user/Embedding-Build-Tools.md‎

Lines changed: 46 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -92,27 +92,34 @@ specified in the plugin configuration.
9292

9393
## Python Dependency Management
9494

95-
The list of third-party Python packages to be downloaded and installed can be specified in Maven or Gradle plugin configuration. Unfortunately,
96-
Python does not enforce strict versioning of dependencies, which can result in problems if a third-party package or one of its transitive
97-
dependencies is unexpectedly updated to a newer version, leading to unforeseen behavior.
95+
for Reproducible Builds
9896

99-
It is regarded as good practice to always specify a Python package with its exact version. In simpler scenarios, where only a few packages
100-
are required, specifying the exact version of each package in the plugin configuration,
101-
along with their transitive dependencies, might be sufficient. However, this method is often impractical,
102-
as manually managing the entire dependency tree can quickly become overwhelming.
97+
In Python ecosystem, it is common that packages specify their dependencies as ranges rather than a fixed version.
98+
For example, package A depends on package B of any version higher or equal to 2.0.0 (denoted as `B>=2.0.0`).
99+
Installation of package A today may pull package `B==2.0.0`. Tomorrow package B releases new version `2.0.1`
100+
and a clean build of a project depending on A will pull new version of B, which may not be compatible
101+
with GraalPy or may introduce (unintentional) breaking changes.
103102

104103
### Locking Dependencies
105104

106-
For these cases, we **highly recommend locking** all Python dependencies whenever there is a change
107-
in the list of required packages for a project. The GraalPy plugins provide an action to do so,
108-
and as a result, a GraalPy lock file will be created, listing all required Python packages with their specific versions
109-
based on the packages defined in the plugin configuration and their dependencies. Subsequent GraalPy plugin executions
110-
will then use this file exclusively to install all packages with guaranteed versions.
105+
We **highly recommend locking** all Python dependencies whenever there is a change in the list of required packages
106+
for a project. Locking the dependencies means explicitly invoking a Maven goal or Gradle task that generates file
107+
`graalpy.lock` that captures versions of all Python package dependencies: those specified explicitly in
108+
`pom.xml` or `build.gradle` and all their transitive dependencies.
111109

112-
The default location of the lock file is in the project root, and since it serves as input for generating resources,
113-
it should be stored alongside other project files in a version control system.
110+
The `graalpy.lock` file should be commited to version control system (e.g., git). Once the `graalpy.lock` file exists,
111+
the package installation during Maven or Gradle build installs the exact same versions as captured in `graalpy.lock`.
114112

115-
For information on the specific Maven or Gradle lock packages actions, please refer to the plugin descriptions below in this document.
113+
When the set of explicit dependencies in `pom.xml` or `build.gradle` changes and does not match what is in
114+
`graalpy.lock` anymore, the build will fail and the user will be asked to explicitly regenerate the `graalpy.lock` file.
115+
116+
Note that unless specific version of a package is desired, we recommend to specify explicit dependencies in
117+
`pom.xml` or `build.gradle` without version quantifier. For some well known packages, GraalPy automatically
118+
installs the version that is known to be compatible with GraalPy. However, once installed, the versions should be
119+
locked to ensure reproducible builds.
120+
121+
For information on the specific Maven or Gradle lock packages actions, please refer to the
122+
Locking Python Packages subsections below.
116123

117124
## GraalPy Maven Plugin
118125

@@ -141,14 +148,6 @@ The Python packages and their versions are specified as if used with `pip`:
141148
...
142149
</configuration>
143150
```
144-
- The **graalPyLockFile** element can specify an alternative path to a GraalPy lock file.
145-
Default value is `${basedir}/graalpy.lock`.
146-
```xml
147-
<configuration>
148-
<graalPyLockFile>${basedir}/graalpy.lock</graalPyLockFile>
149-
...
150-
</configuration>
151-
```
152151

153152
- The **resourceDirectory** element can specify the relative [Java resource path](#java-resource-path).
154153
Remember to use `VirtualFileSystem$Builder#resourceDirectory` when configuring the `VirtualFileSystem` in Java.
@@ -172,9 +171,19 @@ $ mvn org.graalvm.python:graalpy-maven-plugin:lock-packages
172171
```
173172
*Note that the action will override the existing lock file.*
174173

175-
For more information on managing Python packages, please refer to the descriptions of
176-
the `graalPyLockFile` and `packages` fields in the [plugin configuration](#maven-plugin-configuration), as well as the [Python Dependency Management](#python-dependency-management) section
177-
above in this document.
174+
For a high level description of this feature, please refer to the
175+
[Python Dependency Management for Reproducible Builds](#pythop-dependency-management-for-reproducible-builds) section
176+
in this document.
177+
178+
* The **graalPyLockFile** element can change the default path to the GraalPy lock file. Default value is `${basedir}/graalpy.lock`.
179+
The **graalPyLockFile** element by itself will not trigger the locking. The locking must be done by explicitly executing the
180+
`org.graalvm.python:graalpy-maven-plugin:lock-packages` goal.
181+
```xml
182+
<configuration>
183+
<graalPyLockFile>${basedir}/graalpy.lock</graalPyLockFile>
184+
...
185+
</configuration>
186+
```
178187

179188
## GraalPy Gradle Plugin
180189

@@ -202,15 +211,6 @@ The plugin can be configured in the `graalPy` block:
202211
...
203212
}
204213
```
205-
206-
- The **graalPyLockFile** element can specify an alternative path to a GraalPy lock file.
207-
Default value is `$rootDir/graalpy.lock`.
208-
```bash
209-
graalPy {
210-
graalPyLockFile = file("$rootDir/graalpy.lock")
211-
...
212-
}
213-
```
214214

215215
- The **resourceDirectory** element can specify the relative [Java resource path](#java-resource-path).
216216
Remember to use `VirtualFileSystem$Builder#resourceDirectory` when configuring the `VirtualFileSystem` in Java.
@@ -241,10 +241,19 @@ gradle graalPyLockPackages
241241
```
242242
*Note that the action will override the existing lock file.*
243243

244-
For more information on managing Python packages, please refer to the descriptions of
245-
the `graalPyLockFile` and `packages` fields in the [plugin configuration](#gradle-plugin-configuration), as well as the [Python Dependency Management](#python-dependency-management) sections
244+
For a high level description of this feature, please refer to the
245+
[Python Dependency Management for Reproducible Builds](#pythop-dependency-management-for-reproducible-builds) section
246246
in this document.
247247

248+
* The **graalPyLockFile** element can change the default path to the GraalPy lock file. Default value is `${basedir}/graalpy.lock`.
249+
The **graalPyLockFile** element by itself will not trigger the locking. The locking must be done by explicitly executing the
250+
`graalPyLockPackages` task.
251+
```
252+
graalPy {
253+
graalPyLockFile = file("$rootDir/graalpy.lock")
254+
...
255+
}
256+
248257
## Related Documentation
249258
250259
* [Embedding Graal languages in Java](https://www.graalvm.org/reference-manual/embed-languages/)

0 commit comments

Comments
 (0)