Skip to content

Commit 599ee85

Browse files
committed
docs: move contributing instructions from README.md to CONTRIBUTING.md
Signed-off-by: Lalith Suresh <lalith@feldera.com>
1 parent 178ffce commit 599ee85

2 files changed

Lines changed: 52 additions & 53 deletions

File tree

CONTRIBUTING.md

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ notification when you git push.
8787

8888
### Merging a pull request
8989

90-
Since we run benchmarks as part of CI it's good practice to preserve the commit IDs of the feature branch
90+
Since we run benchmarks as part of the CI, it'sa good practice to preserve the commit IDs of the feature branch
9191
we've worked on (and benchmarked). Unfortunately, [the github UI does not have support for this](https://github.com/community/community/discussions/4618)
9292
(it only allows rebase, squash and merge commits to close PRs).
9393
Therefore, it's recommended to merge PRs using the following git CLI invocation:
@@ -98,8 +98,15 @@ git merge --ff-only feature-branch-name
9898
git push upstream main
9999
```
100100

101+
101102
### Code Style
102103

104+
Execute the following command to make `git commit` check the code for formatting issues before commit. It is not yet applied to the sql compiler.
105+
106+
```shell
107+
GITDIR=$(git rev-parse --git-dir)
108+
ln -sf $(pwd)/tools/pre-push ${GITDIR}/hooks/pre-push
109+
```
103110
### Formatting Commit Messages
104111

105112
We follow the conventions on [How to Write a Git Commit Message](http://chris.beams.io/posts/git-commit/).
@@ -111,3 +118,47 @@ and commits.
111118
## Reporting Bugs and Creating Issues
112119

113120
When opening a new issue, try to roughly follow the commit message format conventions above.
121+
122+
123+
# For developers
124+
125+
## Running Benchmarks against DBSP
126+
127+
The repository has a number of benchmarks available in the `benches` directory that provide a comparison of DBSP's performance against a known set of tests.
128+
129+
Each benchmark has its own options and behavior, as outlined below.
130+
131+
### Nexmark Benchmark
132+
133+
You can run the complete set of Nexmark queries, with the default settings, with:
134+
135+
```shell
136+
cargo bench --bench nexmark --features with-nexmark
137+
```
138+
139+
By default this will run each query with a total of 100 million events emitted at 10M per second (by two event generator threads), using 2 CPU cores for processing the data.
140+
141+
To run just the one query, q3, with only 10 million events, but using 8 CPU cores to process the data and 6 event generator threads, you can run:
142+
143+
```shell
144+
cargo bench --bench nexmark --features with-nexmark -- --query q3 --max-events 10000000 --cpu-cores 8 --num-event-generators 6
145+
```
146+
147+
For further options that you can use with the Nexmark benchmark,
148+
149+
```shell
150+
cargo bench --bench nexmark --features with-nexmark -- --help
151+
```
152+
153+
An extensive blog post about the implementation of Nexmark in DBSP:
154+
<https://liveandletlearn.net/post/vmware-take-3-experience-with-rust-and-dbsp/>
155+
156+
157+
## Updating the pipeline manager database schema
158+
159+
Here are some guidelines when contributing code that affects the Pipeline Manager's DB schema.
160+
161+
* We use SQL migrations to apply the schema to a live database to faciliate upgrades. We use [refinery](https://github.com/rust-db/refinery) to manage migrations.
162+
* The migration files can be found in `crates/pipeline_manager/migrations`
163+
* Do not modify an existing migration file. If you want to evolve the schema, add a new SQL or rust file to the migrations folder following [refinery's versioning and naming scheme](https://docs.rs/refinery/latest/refinery/#usage). The migration script should update an existing schema as opposed to assuming a clean slate. For example, use `ALTER TABLE` to add a new column to an existing table and fill that column for existing rows with the appropriate defaults.
164+
* If you add a new migration script `V{i}`, add tests for migrations from `V{i-1} to V{i}`. For example, add tests that invoke the pipeline manager APIs before and after the migration.

README.md

Lines changed: 0 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -161,55 +161,3 @@ To learn about using DBSP as a Rust programmer, start by reading the
161161
[`circuit_builder`](`circuit::circuit_builder`) module documentation,
162162
or the examples folder. For more sophisticated examples, try looking
163163
at the `nexmark` benchmark in the `benches` directory.
164-
165-
# Contributing
166-
167-
## Setting up git hooks
168-
169-
Execute the following command to make `git commit` check the code before commit:
170-
171-
```shell
172-
GITDIR=$(git rev-parse --git-dir)
173-
ln -sf $(pwd)/tools/pre-push ${GITDIR}/hooks/pre-push
174-
```
175-
176-
## Running Benchmarks against DBSP
177-
178-
The repository has a number of benchmarks available in the `benches` directory that provide a comparison of DBSP's performance against a known set of tests.
179-
180-
Each benchmark has its own options and behavior, as outlined below.
181-
182-
### Nexmark Benchmark
183-
184-
You can run the complete set of Nexmark queries, with the default settings, with:
185-
186-
```shell
187-
cargo bench --bench nexmark --features with-nexmark
188-
```
189-
190-
By default this will run each query with a total of 100 million events emitted at 10M per second (by two event generator threads), using 2 CPU cores for processing the data.
191-
192-
To run just the one query, q3, with only 10 million events, but using 8 CPU cores to process the data and 6 event generator threads, you can run:
193-
194-
```shell
195-
cargo bench --bench nexmark --features with-nexmark -- --query q3 --max-events 10000000 --cpu-cores 8 --num-event-generators 6
196-
```
197-
198-
For further options that you can use with the Nexmark benchmark,
199-
200-
```shell
201-
cargo bench --bench nexmark --features with-nexmark -- --help
202-
```
203-
204-
An extensive blog post about the implementation of Nexmark in DBSP:
205-
<https://liveandletlearn.net/post/vmware-take-3-experience-with-rust-and-dbsp/>
206-
207-
208-
## Updating the pipeline manager database schema
209-
210-
Here are some guidelines when contributing code that affects the Pipeline Manager's DB schema.
211-
212-
* We use SQL migrations to apply the schema to a live database to faciliate upgrades. We use [refinery](https://github.com/rust-db/refinery) to manage migrations.
213-
* The migration files can be found in `crates/pipeline_manager/migrations`
214-
* Do not modify an existing migration file. If you want to evolve the schema, add a new SQL or rust file to the migrations folder following [refinery's versioning and naming scheme](https://docs.rs/refinery/latest/refinery/#usage). The migration script should update an existing schema as opposed to assuming a clean slate. For example, use `ALTER TABLE` to add a new column to an existing table and fill that column for existing rows with the appropriate defaults.
215-
* If you add a new migration script `V{i}`, add tests for migrations from `V{i-1} to V{i}`. For example, add tests that invoke the pipeline manager APIs before and after the migration.

0 commit comments

Comments
 (0)