You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+44-5Lines changed: 44 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -87,7 +87,7 @@ notification when you git push.
87
87
88
88
### Merging a pull request
89
89
90
-
Since we run benchmarks as part of the CI, it'sa good practice to preserve the commit IDs of the feature branch
90
+
Since we run benchmarks as part of the CI, it's a good practice to preserve the commit IDs of the feature branch
91
91
we've worked on (and benchmarked). Unfortunately, [the github UI does not have support for this](https://github.com/community/community/discussions/4618)
92
92
(it only allows rebase, squash and merge commits to close PRs).
93
93
Therefore, it's recommended to merge PRs using the following git CLI invocation:
@@ -101,7 +101,7 @@ git push upstream main
101
101
102
102
### Code Style
103
103
104
-
Execute the following command to make `git commit` check the code for formatting issues before commit. It is not yet applied to the sql compiler.
104
+
Execute the following command to make `git push` check the code for formatting issues.
105
105
106
106
```shell
107
107
GITDIR=$(git rev-parse --git-dir)
@@ -122,6 +122,44 @@ When opening a new issue, try to roughly follow the commit message format conven
122
122
123
123
# For developers
124
124
125
+
## Building DBSP from sources
126
+
127
+
DBSP is implemented in Rust and uses Rust's `cargo` build system. The SQL
128
+
to DBSP compiler is implemented in Java and uses `maven` as its build system.
129
+
130
+
You can build the rust sources by runnning the following at the top level of this tree.
131
+
132
+
```
133
+
cargo build
134
+
```
135
+
136
+
To build the SQL to DBSP compiler, run the following from `sql-to-dbsp-compiler/SQL-compiler`:
137
+
138
+
```
139
+
mvn package
140
+
```
141
+
142
+
If you want to develop DBSP without installing the required toolchains
143
+
locally, you can use Github Codespaces; from
144
+
https://github.com/feldera/dbsp, click on the green `<> Code` button,
145
+
then select Codespaces and click on "Create codespace on main".
146
+
147
+
## Learning the DBSP Rust code
148
+
149
+
To learn how the DBSP core works, we recommend starting with the tutorial.
150
+
151
+
From the project root:
152
+
153
+
```
154
+
cargo doc --open
155
+
```
156
+
157
+
Then search for `dbsp::tutorial`.
158
+
159
+
Another good place to start is the `circuit::circuit_builder` module documentation,
160
+
or the examples folder. For more sophisticated examples, try looking
161
+
at the `nexmark` benchmark in the `benches` directory.
162
+
125
163
## Running Benchmarks against DBSP
126
164
127
165
The repository has a number of benchmarks available in the `benches` directory that provide a comparison of DBSP's performance against a known set of tests.
@@ -156,9 +194,10 @@ An extensive blog post about the implementation of Nexmark in DBSP:
156
194
157
195
## Updating the pipeline manager database schema
158
196
159
-
Here are some guidelines when contributing code that affects the Pipeline Manager's DB schema.
197
+
The pipeline manager serves as the API server for Feldera. It persists API state in a Postgres DB instance.
198
+
Here are some guidelines when contributing code that affects this database's schema.
160
199
161
-
* We use SQL migrations to apply the schema to a live database to faciliate upgrades. We use [refinery](https://github.com/rust-db/refinery) to manage migrations.
200
+
* We use SQL migrations to apply the schema to a live database to facilitate upgrades. We use [refinery](https://github.com/rust-db/refinery) to manage migrations.
162
201
* The migration files can be found in `crates/pipeline_manager/migrations`
163
202
* Do not modify an existing migration file. If you want to evolve the schema, add a new SQL or rust file to the migrations folder following [refinery's versioning and naming scheme](https://docs.rs/refinery/latest/refinery/#usage). The migration script should update an existing schema as opposed to assuming a clean slate. For example, use `ALTER TABLE` to add a new column to an existing table and fill that column for existing rows with the appropriate defaults.
164
-
* If you add a new migration script `V{i}`, add tests for migrations from `V{i-1} to V{i}`. For example, add tests that invoke the pipeline manager APIs before and after the migration.
203
+
* If you add a new migration script `V{i}`, add tests for migrations from `V{i-1}` to `V{i}`. For example, add tests that invoke the pipeline manager APIs before and after the migration.
0 commit comments