Gammatone learnable filterbanks#2714
Conversation
- fix transpose in gammatoneconv
…matone filter bank
|
Original (tensorflow only) paper implementation available for ref. at https://github.com/epfl-lts2/learnable-filterbanks/ |
|
Hi and thank you for your contribution! Could you please add results to the readme? It is important to know what numbers you obtain with this code compared to what already exists. Also, please fix the failing tests as a first step. Feel free to install pre-commit locally to facilitate the exercice. |
|
@TParcollet Hello! I had run the pre-commit but I merged the develop branch back on github so it broke a couple of things, I will fix that asap. I have not implemented (yet) the recipes to reproduce our paper, but replacing the LEAF frontend (which includes a more sophisticated learnable pooling and compression) by our Gammatone LFB on the google speech commands dataset yields a test accuracy of 96.5% (vs 97.4% for LEAF). Should I include this result too ? |
f015bc1 to
7b59d71
Compare
* data prep scripts update * iterate over utterances * without parallel map * parallel map -> so fast omfg * gigaspeech data prep done * speechcolab extra dep if one must download gigaspeech * create ASR CTC folder * base yaml + update data prep to better reflect potential different naming for csvs * update recipe * update recipe to be compliant with gigaspeech csv * add transformers dep * convert opus to wav * recipe --debug mode works. * typo GRABAGE_UTTERANCE_TAGS -> GARBAGE_UTTERANCE_TAGS * tmp DL file * update DL FILE * add DL file in ASR/CTC * update extra_requirements.txt * add support of savedir within Pretrained subclasses * add wbs requirements * webdataset * remove print * tmp files webdataset * verbosity + metada.json * letzo now label_encoder can actually train + the recipe seems to work. * remove wbs * DL info * HF DL support * remove webdataset as it sucks :p * name * ngram commands * whisper baseline * fix HF * pre-commit + sentencepiece char * remove csv * Add quirks.py, move global PyTorch config and GPU workarounds there * Add support for SB_DISABLE_QUIRKS environment variable * Fetch rework: make savedir optional * bunch of updates to make it run * no download script * fix precommit * fix precommit * readmes * readmes * readmes * readmes * doc update * CI god not happy, make CI god happy * why you here little encoder * adding a tranduscer streaming recipe, because why not * add test for transducer * works better when me not stupid * fix yaml * update req * add warning for cache dir * add warning for cache dir * enable multiprocessing * Minor cleanups to fetching * Change default behavior of inference to not create savedir if not specified * allow data prep without ddp * fix tests * smoll readme update * fix review comments * fixed concat_start_index check (speechbrain#2717) * Ensure adapted models save their parameters (speechbrain#2716) Co-authored-by: Parcollet Titouan <parcollet.titouan@gmail.com> * wtf * update doc * more documentation on storage * missing arg * a bit of logs * new schedulers * new schedulers * Fixes speechbrain#2656: Remove EOS from SoundChoice * fix my stupidity * Update non-HF code path for new preprocessing code in GigaSpeech * Fix CSV path for non-HF Gigaspeech * Fix formatting * Kmeans fix (speechbrain#2642) * fix kmeans bug * fix final batch * fix chuncksize * fix * fix * fix precommit * fix doxstrin inconsistency * fix precommit * fix doc string --------- Co-authored-by: Mirco Ravanelli <mirco.ravanelli@gmail.com> * add call on start of fit_batch fn * Update core.py Fix old commit * Update core.py * Fix preprocess_text example * Fix guess_source docstring with up-to-date info * Also remove default savedir from Pretrained * Fix function name for log_applied_quirks * wip audiomnist+gt * Revert "fix normalization for LFB" This reverts commit 3fd0330. * audiomnist classification setup * fix config * add missing file * update dataset load/training * remove unnecessary params * remove sort * remove unnecessary code * fix paths * fix loss computation * add missing flatten * print summary * Explain quirks in docs/experiment.md * ok stupid linter check that hates intentional leading spaces in markdown * add citing in README * add code to pad all wavs to the same length * fix pad call * fix error computation * fix error computation * Make `collect_in` optional for `Pretrainer`, disable it by default * Change more defaults to `savedir=None` and `fetch_strategy=SYMLINK` Since the SYMLINK strategy falls back to NO_LINK whenever `savedir is None`, it makes sense to switch more things to default to `savedir=None`. Should the `savedir` explicitly be set by the user, past behavior is preserved (defaulting to symlinks). * move flatten in audionet * Fix GS transducer test prediction decoding? * fix data prep logic and paths * Actually fix GS transducer test prediction decoding * Remove punctuation filtering that is handled elsewhere * HuggingFance * fix skip data prep logic * add original audionet feature extraction * fix pooling for audionet feature extraction * fix audionet shape + remove input norm * try data augmentation * add missing refs * - rework AudioNet to have optional pooling - use official AudioMNIST train/test/valid splits * fix typo in url * update audionet hparams * update audionet custom hparams * update audionet custom hparams * Updated warning for load_collected * Add results and notices for results for GigaSpeech transducer & wavlm * english hard * update audionet custom hparams * fix doc + pre-commit clean * fix code examples * fix consistency tests * fix pre commit * remove config * fix docstring for LFB * fix docstring for GammatoneConv1D --------- Co-authored-by: Adel Moumen <adelmoumen.pro@gmail.com> Co-authored-by: Adel Moumen <88119391+Adel-Moumen@users.noreply.github.com> Co-authored-by: asu <sdelang@sdelang.fr> Co-authored-by: TParcollet <parcollet.titouan@gmail.com> Co-authored-by: Peter Plantinga <plantinga.peter@proton.me> Co-authored-by: gianfranco <62777451+gfdb@users.noreply.github.com> Co-authored-by: Peter Plantinga <plantinga.peter@protonmail.com> Co-authored-by: Titouan Parcollet/Embedded AI /SRUK/Engineer/Samsung Electronics <t.parcollet@sruk-ccn4.eu.corp.samsungelectronics.net> Co-authored-by: flexthink <flexthink@users.noreply.github.com> Co-authored-by: Pooneh Mousavi <moosavi.pooneh@gmail.com> Co-authored-by: Mirco Ravanelli <mirco.ravanelli@gmail.com>
|
@TParcollet Hi. I added the results for the AudioMNIST classification task, which are consistent with the ones we got in our paper. The tests have also been fixed. Unless you have concerns about those changes, I will work to add the Google speech command experiment we did. |
|
I have checked the build failure, but it seems related to huggingface stuff, not sure I can fix this... |
* update README * specify encoding for open()
|
I doubled check on my fork and all builds are passing (after merging back develop) |
merge develop changes
Gammatone dev
|
I wonder if anyone will review this PR... oh well at least I tried. |
Hey @naspert, sorry for the delay! We apologize as we had quite a lot of work to do and weren't able to process this PR on time. @TParcollet, what your current views on this PR? |
|
@Adel-Moumen thanks for your answer (and I know maintainers are volunteers that do this most necessary work on their spare time) ! And if anybody has feedback, that would be very welcome :) |
What does this PR do?
This PR implements parametric Gammatone filterbank, as proposed in the paper "Learnable filter-banks
for CNN-based audio applications", in Proc. of NLDL 2022, initially proposed for ICLR 2020
This is similar to the LEAF frontend already included in the library (except LEAF uses complex Gabor instead of Gammatones). The interest of Gammatone filter banks is that they are considered to be good models of the human ear.
Before submitting
PR review
Reviewer checklist