You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+52-1Lines changed: 52 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -87,7 +87,7 @@ notification when you git push.
87
87
88
88
### Merging a pull request
89
89
90
-
Since we run benchmarks as part of CI it's good practice to preserve the commit IDs of the feature branch
90
+
Since we run benchmarks as part of the CI, it'sa good practice to preserve the commit IDs of the feature branch
91
91
we've worked on (and benchmarked). Unfortunately, [the github UI does not have support for this](https://github.com/community/community/discussions/4618)
92
92
(it only allows rebase, squash and merge commits to close PRs).
93
93
Therefore, it's recommended to merge PRs using the following git CLI invocation:
We follow the conventions on [How to Write a Git Commit Message](http://chris.beams.io/posts/git-commit/).
@@ -111,3 +118,47 @@ and commits.
111
118
## Reporting Bugs and Creating Issues
112
119
113
120
When opening a new issue, try to roughly follow the commit message format conventions above.
121
+
122
+
123
+
# For developers
124
+
125
+
## Running Benchmarks against DBSP
126
+
127
+
The repository has a number of benchmarks available in the `benches` directory that provide a comparison of DBSP's performance against a known set of tests.
128
+
129
+
Each benchmark has its own options and behavior, as outlined below.
130
+
131
+
### Nexmark Benchmark
132
+
133
+
You can run the complete set of Nexmark queries, with the default settings, with:
By default this will run each query with a total of 100 million events emitted at 10M per second (by two event generator threads), using 2 CPU cores for processing the data.
140
+
141
+
To run just the one query, q3, with only 10 million events, but using 8 CPU cores to process the data and 6 event generator threads, you can run:
Here are some guidelines when contributing code that affects the Pipeline Manager's DB schema.
160
+
161
+
* We use SQL migrations to apply the schema to a live database to faciliate upgrades. We use [refinery](https://github.com/rust-db/refinery) to manage migrations.
162
+
* The migration files can be found in `crates/pipeline_manager/migrations`
163
+
* Do not modify an existing migration file. If you want to evolve the schema, add a new SQL or rust file to the migrations folder following [refinery's versioning and naming scheme](https://docs.rs/refinery/latest/refinery/#usage). The migration script should update an existing schema as opposed to assuming a clean slate. For example, use `ALTER TABLE` to add a new column to an existing table and fill that column for existing rows with the appropriate defaults.
164
+
* If you add a new migration script `V{i}`, add tests for migrations from `V{i-1} to V{i}`. For example, add tests that invoke the pipeline manager APIs before and after the migration.
The repository has a number of benchmarks available in the `benches` directory that provide a comparison of DBSP's performance against a known set of tests.
179
-
180
-
Each benchmark has its own options and behavior, as outlined below.
181
-
182
-
### Nexmark Benchmark
183
-
184
-
You can run the complete set of Nexmark queries, with the default settings, with:
By default this will run each query with a total of 100 million events emitted at 10M per second (by two event generator threads), using 2 CPU cores for processing the data.
191
-
192
-
To run just the one query, q3, with only 10 million events, but using 8 CPU cores to process the data and 6 event generator threads, you can run:
Here are some guidelines when contributing code that affects the Pipeline Manager's DB schema.
211
-
212
-
* We use SQL migrations to apply the schema to a live database to faciliate upgrades. We use [refinery](https://github.com/rust-db/refinery) to manage migrations.
213
-
* The migration files can be found in `crates/pipeline_manager/migrations`
214
-
* Do not modify an existing migration file. If you want to evolve the schema, add a new SQL or rust file to the migrations folder following [refinery's versioning and naming scheme](https://docs.rs/refinery/latest/refinery/#usage). The migration script should update an existing schema as opposed to assuming a clean slate. For example, use `ALTER TABLE` to add a new column to an existing table and fill that column for existing rows with the appropriate defaults.
215
-
* If you add a new migration script `V{i}`, add tests for migrations from `V{i-1} to V{i}`. For example, add tests that invoke the pipeline manager APIs before and after the migration.
0 commit comments