Skip to content

Commit 0dbaf79

Browse files
TParcolletCheeky Sparrow
andauthored
Adopt pyproject and move to Ruff (#2946)
Co-authored-by: Cheeky Sparrow <ck@ck.com>
1 parent 7d214a3 commit 0dbaf79

353 files changed

Lines changed: 5743 additions & 2553 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.flake8

Lines changed: 0 additions & 8 deletions
This file was deleted.

.pre-commit-config.yaml

Lines changed: 5 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -13,22 +13,14 @@ repos:
1313
- id: check-added-large-files
1414
args: [--maxkb=1024]
1515

16-
- repo: https://github.com/psf/black
17-
rev: 24.3.0
16+
- repo: https://github.com/astral-sh/ruff-pre-commit
17+
rev: v0.12.4
1818
hooks:
19-
- id: black
19+
- id: ruff
2020
types: [python]
21-
additional_dependencies: ['click==8.1.7']
22-
23-
- repo: https://github.com/PyCQA/flake8
24-
rev: 7.0.0
25-
hooks:
26-
- id: flake8
21+
args: [--fix]
22+
- id: ruff-format
2723
types: [python]
28-
additional_dependencies:
29-
- pydoclint==0.4.1
30-
- pycodestyle==2.11.0
31-
- flake8-encodings==0.5.1
3224

3325
- repo: https://github.com/adrienverge/yamllint
3426
rev: v1.35.1
@@ -50,8 +42,3 @@ repos:
5042
- "--ignore-regex='base64,.*?=='"
5143
additional_dependencies:
5244
- tomli
53-
54-
- repo: https://github.com/pycqa/isort
55-
rev: 5.13.2
56-
hooks:
57-
- id: isort

conftest.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ def pytest_generate_tests(metafunc):
1111

1212

1313
collect_ignore = [
14-
"setup.py",
1514
"speechbrain/integrations/",
1615
# These can be removed once the modules are fully deprecated
1716
"speechbrain/utils/bleu.py",

docs/contributing.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,8 @@ We target a specific range of supported Python versions, which are tested via CI
5252

5353
### Formatting & linting
5454

55-
Use `pre-commit run -a` to run formatting and linting, using tools like `black`
56-
and `flake8` under the hood (see [`.pre-commit-config.yaml`](../.pre-commit-config.yaml)).
55+
Use `pre-commit run -a` to run formatting and linting, using tools like `ruff`
56+
under the hood (see [`.pre-commit-config.yaml`](../.pre-commit-config.yaml)).
5757
Some passes automatically fix your code, and some may require your intervention.
5858

5959
These checks are run and enforced on the CI.

docs/coverage.md

Lines changed: 4 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,8 @@ our tests cover one PyTorch version only, _the latest_.
4545
* hook: https://github.com/psf/black
4646
<br/> black
4747
<br/> click
48-
* hook: https://gitlab.com/pycqa/flake8.git
49-
<br/> flake8; see: [.flake8](https://github.com/speechbrain/speechbrain/blob/develop/.flake8)
48+
* hook: https://github.com/astral-sh/ruff-pre-commit
49+
<br/> ruff; see: [pyproject.toml](https://github.com/speechbrain/speechbrain/blob/develop/pyproject.toml) for configuration
5050
* hook: https://github.com/adrienverge/yamllint
5151
<br/> yamllint; see: [.yamllint.yaml](https://github.com/speechbrain/speechbrain/blob/develop/.yamllint.yaml)
5252

@@ -128,8 +128,7 @@ our tests cover one PyTorch version only, _the latest_.
128128
* [LICENSE](https://github.com/speechbrain/speechbrain/blob/develop/LICENSE)
129129
* [README.md](https://github.com/speechbrain/speechbrain/blob/develop/README.md)
130130
* [pyproject.toml](https://github.com/speechbrain/speechbrain/blob/develop/pyproject.toml) - target-version = ['py38']
131-
* [setup.py](https://github.com/speechbrain/speechbrain/blob/develop/setup.py)
132-
* python_requires=">=3.7",
131+
* python_requires=">=3.8.1",
133132
* uses: [speechbrain/version.txt](https://github.com/speechbrain/speechbrain/blob/develop/speechbrain/version.txt)
134133
* requires:
135134
```
@@ -206,22 +205,7 @@ speechbrain/lm/ngram.py 36 1 97%
206205
speechbrain/lobes/augment.py 154 55 64% <== < 80%
207206
speechbrain/lobes/beamform_multimic.py 20 14 30% <== < 80%
208207
speechbrain/lobes/features.py 96 9 91%
209-
speechbrain/lobes/models/CRDNN.py 52 12 77% <== < 80%
210-
speechbrain/lobes/models/ContextNet.py 83 3 96%
211-
speechbrain/lobes/models/ECAPA_TDNN.py 157 7 96%
212-
speechbrain/lobes/models/HifiGAN.py 321 146 55% <== < 80%
213-
speechbrain/lobes/models/MetricGAN.py 74 29 61% <== < 80%
214-
speechbrain/lobes/models/Tacotron2.py 364 66 82%
215-
speechbrain/lobes/models/conv_tasnet.py 121 6 95%
216-
speechbrain/lobes/models/dual_path.py 357 55 85%
217-
speechbrain/lobes/models/fairseq_wav2vec.py 93 93 0% <== < 80%
218-
speechbrain/lobes/models/g2p/dataio.py 136 107 21% <== < 80%
219-
speechbrain/lobes/models/g2p/homograph.py 118 20 83%
220-
speechbrain/lobes/models/g2p/model.py 132 109 17% <== < 80%
221-
speechbrain/lobes/models/huggingface_wav2vec.py 145 47 68% <== < 80%
222-
speechbrain/lobes/models/resepformer.py 180 21 88%
223-
speechbrain/lobes/models/segan_model.py 102 88 14% <== < 80%
224-
speechbrain/lobes/models/transformer/Conformer.py 111 7 94%
208+
speechbrain/lobes/models/CRDNN.py 52 12 77% <== < 80% Ruff #29
225209
speechbrain/lobes/models/transformer/Transformer.py 180 22 88%
226210
speechbrain/lobes/models/transformer/TransformerASR.py 92 28 70% <== < 80%
227211
speechbrain/lobes/models/transformer/TransformerLM.py 47 5 89%

docs/devtools.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,31 +2,31 @@
22

33
## Linting/formatting/testing
44

5-
### flake8
6-
- A bit like pycodestyle: make sure the codestyle is according to guidelines.
7-
- Compatible with black, in fact, current flake8 config directly taken from black
8-
- Code compliance can be tested simply with: `flake8 <file-or-directory>`
9-
- You can bypass flake8 for a line with `# noqa: <QA-CODE> E.G. # noqa: E731 to allow lambda assignment`
5+
### ruff
6+
- A fast Python linter and formatter that replaces multiple tools like flake8, isort, and pycodestyle
7+
- Compatible with black and provides similar functionality to flake8
8+
- Code compliance can be tested simply with: `ruff check <file-or-directory>`
9+
- You can bypass ruff for a line with `# noqa: <RULE-CODE>` E.G. `# noqa: E731` to allow lambda assignment
10+
- Can also format code with: `ruff format <file-or-directory>`
1011

1112
### pre-commit
1213
- Python tool which takes a configuration file (.pre-commit-config.yaml) and installs the git commit hooks specified in it.
1314
- Git commit hooks are local so all who want to use them need to install them separately. This is done by: `pre-commit install`
1415
- The tool can also install pre-push hooks. This is done separately with: `pre-commit install --hook-type pre-push --config .pre-push-config.yaml`
1516

1617
### the git pre-commit hooks
17-
- Automatically run black
18+
- Automatically run ruff (linting and formatting)
1819
- Automatically fix trailing whitespace, end of file, sort requirements.txt
1920
- Check that no large (>512kb) files are added by accident
20-
- Automatically run flake8
2121
- Automatically run cspell
22-
- NOTE: If the hooks fix something (e.g. trailing whitespace or reformat with black), these changes are not automatically added and committed. Youll have to add the fixed files again and run the commit again. I guess this is a safeguard: dont blindly accept changes from git hooks.
22+
- NOTE: If the hooks fix something (e.g. trailing whitespace or reformat with ruff), these changes are not automatically added and committed. You'll have to add the fixed files again and run the commit again. I guess this is a safeguard: don't blindly accept changes from git hooks.
2323
- NOTE2: The hooks are only run on the files you git added to the commit. This is in contrast to the CI pipeline, which always tests everything.
2424
- NOTE3: If a word is flagged as a spelling error but it should be kept, you can add the word to `.dict-speechbrain.txt`
2525

2626
### the git pre-push hooks
27-
- Black and flake8 as checks on the whole repo
27+
- Ruff as checks on the whole repo
2828
- Unit-tests and doctests run on the whole repo
29-
- These hooks can only be run in the full environment, so if you install these, youll need to e.g. activate virtualenv before pushing.
29+
- These hooks can only be run in the full environment, so if you install these, you'll need to e.g. activate virtualenv before pushing.
3030

3131
### pytest doctests
3232
- This is not an additional dependency, but just that doctests are now run with pytest. Use: `pytest --doctest-modules <file-or-directory>`
@@ -46,8 +46,8 @@
4646
- CD stands for continuous deployment, check out the "Releasing a new version" section.
4747

4848
### Our test suite
49-
- Code linters are run. This means black and flake8. These are run on everything in speechbrain (the library directory), everything in recipes and everything in tests.
50-
- Note that black will only error out if it would change a file here, but wont reformat anything at this stage. Youll have to run black on your code and push a new commit. The black commit hook helps avoid these errors.
49+
- Code linters are run. This means ruff. These are run on everything in speechbrain (the library directory), everything in recipes and everything in tests.
50+
- Note that ruff will only error out if it would change a file here, but won't reformat anything at this stage. You'll have to run ruff on your code and push a new commit. The ruff commit hook helps avoid these errors.
5151
- All unit-tests and doctests are run. You can check that these pass by running them yourself before pushing, with `pytest tests` and `pytest --doctest-modules speechbrain`
5252
- Integration tests (minimal examples). The minimal examples serve both to
5353
illustrate basic tasks and experiment running, but also as integration tests

docs/installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ SpeechBrain is constantly evolving. New features, tutorials, and documentation w
55

66
SpeechBrain supports both CPU and GPU computation. For most recipes, however, a GPU is necessary during training. Please note that CUDA must be properly installed to use GPUs.
77

8-
We support [PyTorch](https://pytorch.org/get-started/locally/) 1.9+ and Python 3.8-3.12 (newer Python versions than advertised may work if supported by PyTorch).
8+
We support [PyTorch](https://pytorch.org/get-started/locally/) 1.9+ and Python 3.8.1-3.12 (newer Python versions than advertised may work if supported by PyTorch).
99
We recommend you upgrade to at least 3.9+ as support for 3.8 will be removed eventually.
1010

1111
## Install via PyPI

lint-requirements.txt

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,4 @@
1-
black==24.3.0
2-
click==8.1.7
3-
flake8==7.0.0
4-
isort==5.13.2
5-
pycodestyle==2.11.0
6-
pydoclint==0.4.1
1+
codespell>=2.3.0
72
pytest==7.4.0
3+
ruff==v0.12.4
84
yamllint==1.35.1

pyproject.toml

Lines changed: 125 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,129 @@
1-
[tool.black]
1+
[build-system]
2+
requires = ["setuptools>=61.0", "wheel"]
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "speechbrain"
7+
dynamic = ["version"]
8+
description = "All-in-one speech toolkit in pure Python and Pytorch"
9+
readme = "README.md"
10+
license = {text = "Apache-2.0"}
11+
authors = [
12+
{name = "Mirco Ravanelli, Titouan Parcollet, Adel Moumen, Sylvain de Langen, Cem Subakan, Peter Plantinga, Yingzhi Wang, Pooneh Mousavi, Luca Della Libera, Artem Ploujnikov, Francesco Paissan, Davide Borra, Salah Zaiem, Zeyu Zhao, Shucong Zhang, Georgios Karakasidis, Sung-Lin Yeh, Pierre Champion, Aku Rouhe, Rudolf Braun, Florian Mai, Juan Zuluaga-Gomez, Seyed Mahed Mousavi, Andreas Nautsch, Ha Nguyen, Xuechen Liu, Sangeet Sagar, Jarod Duret, Salima Mdhaffar, Gaelle Laperriere, Mickael Rouvier, Renato De Mori, Yannick Esteve"}
13+
]
14+
classifiers = [
15+
"Programming Language :: Python :: 3",
16+
"License :: OSI Approved :: Apache Software License",
17+
]
18+
requires-python = ">=3.8.1"
19+
dependencies = [
20+
"hyperpyyaml",
21+
"joblib",
22+
"numpy",
23+
"packaging",
24+
"scipy",
25+
"sentencepiece",
26+
"torch>=1.9",
27+
"torchaudio",
28+
"tqdm",
29+
"huggingface_hub",
30+
]
31+
keywords = ["speech", "audio", "pytorch", "deep-learning"]
32+
33+
[project.urls]
34+
Homepage = "https://speechbrain.github.io/"
35+
36+
[project.optional-dependencies]
37+
dev = [
38+
"ruff==0.12.4",
39+
"pytest==7.4.0",
40+
"yamllint==1.35.1",
41+
"pre-commit>=2.3.0",
42+
"pandas>=1.0.1",
43+
"transformers>=4.30.0",
44+
"codespell>=2.3.0",
45+
]
46+
47+
[tool.setuptools.dynamic]
48+
version = {file = "speechbrain/version.txt"}
49+
50+
[tool.setuptools.packages.find]
51+
exclude = ["tests", "tests.*"]
52+
53+
[tool.setuptools.package-data]
54+
speechbrain = ["version.txt", "log-config.yaml"]
55+
56+
[tool.ruff]
57+
target-version = "py38"
258
line-length = 80
3-
target-version = ['py38']
4-
exclude = '''
5-
6-
(
7-
/(
8-
\.eggs # exclude a few common directories in the
9-
| \.git # root of the project
10-
| \.mypy_cache
11-
| \.tox
12-
| \.venv
13-
)/
14-
)
15-
'''
59+
exclude = [
60+
".eggs",
61+
".git",
62+
".mypy_cache",
63+
".tox",
64+
".venv",
65+
"tests/tmp",
66+
]
67+
68+
[tool.ruff.lint]
69+
select = [
70+
"E", # pycodestyle errors
71+
"W", # pycodestyle warnings
72+
"F", # pyflakes
73+
"I", # isort (import sorting)
74+
"B", # flake8-bugbear
75+
"C4", # flake8-comprehensions
76+
"UP", # pyupgrade
77+
]
78+
ignore = [
79+
"B007", # Loop control variable not used within loop body
80+
"B904", # Within an `except` clause, raise exceptions with `raise ... from err`
81+
"B028", # No explicit `stacklevel` keyword argument found
82+
"C400", # Unnecessary generator (rewrite as a `list` comprehension)
83+
"C401", # Unnecessary generator (rewrite as a `set` comprehension)
84+
"C403", # Unnecessary `list` comprehension (rewrite as a `set` comprehension)
85+
"C404", # Unnecessary `list` call within `sorted()`
86+
"C405", # Unnecessary `list` literal (rewrite as a `set` literal)
87+
"C408", # Unnecessary `dict`/`list` call (rewrite as literal)
88+
"C409", # Unnecessary `tuple` literal passed to `tuple()`
89+
"C414", # Unnecessary `list` call within `sorted()`
90+
"C416", # Unnecessary `list` comprehension (rewrite using `list()`)
91+
"C417", # Unnecessary `map` usage (rewrite using a `list` comprehension)
92+
"C419", # Unnecessary list comprehension
93+
"E203", # whitespace before ':' (black compatibility)
94+
"E266", # too many leading '#' for block comment
95+
"E501", # line too long (handled by formatter)
96+
"E721", # Do not compare types, use `isinstance()`
97+
"F601", # Dictionary key literal repeated
98+
"UP008", # Use `super()` instead of `super(__class__, self)`
99+
"UP028", # Replace `yield` over `for` loop with `yield from`
100+
"UP030", # Use implicit references for positional format fields
101+
"UP031", # Use format specifiers instead of percent format
102+
"B008", # do not perform function calls in argument defaults
103+
"B006", # do not use mutable data structures for argument defaults
104+
"B011", # Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
105+
"B019", # Use of `functools.lru_cache` or `functools.cache` on methods can lead to memory leaks
106+
"B023", # Function definition does not bind loop variable
107+
"B026", # Star-arg unpacking after a keyword argument is strongly discouraged
108+
]
109+
110+
[tool.ruff.lint.isort]
111+
# Import sorting configuration (black-compatible)
112+
combine-as-imports = true
113+
force-wrap-aliases = true
114+
known-first-party = ["speechbrain"]
115+
known-third-party = ["torch", "torchaudio", "numpy", "scipy", "hyperpyyaml", "joblib", "packaging", "sentencepiece", "tqdm", "huggingface_hub"]
116+
split-on-trailing-comma = false
117+
118+
[tool.ruff.lint.per-file-ignores]
119+
"__init__.py" = ["F401"]
120+
121+
[tool.ruff.format]
122+
quote-style = "double"
123+
indent-style = "space"
124+
skip-magic-trailing-comma = false
125+
line-ending = "auto"
126+
docstring-code-format = true
16127

17128
[tool.codespell]
18129
skip = "./tests/tmp,./**/result,*.csv,*train.txt,*test.txt"
19-
20-
[tool.isort]
21-
profile = "black"
22-
line_length = 80
23-
filter_files = true

recipes/AISHELL-1/aishell_prepare.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ def prepare_aishell(
150150
data_folder, "transcript/aishell_transcript_v0.8.txt"
151151
)
152152

153-
with open(path_to_transcript, "r", encoding="utf-8") as f:
153+
with open(path_to_transcript, encoding="utf-8") as f:
154154
lines = f.readlines()
155155
for line in lines:
156156
key = line.split()[0]

0 commit comments

Comments
 (0)