Conversation
…peechbrain into whisper-finetuning
| enc_states, memory | ||
| ) # TODO: switch args | ||
| log_probs = self.softmax(dec_out[:, -1]) | ||
| return log_probs, memory, None |
There was a problem hiding this comment.
Are we adding an argument here?
| dec_out = self.model.forward_decoder(enc_states, memory) | ||
| dec_out, attn, = self.module.forward_decoder(enc_states, memory) | ||
| log_probs = self.softmax(dec_out[:, -1]) | ||
| return log_probs, memory, None |
There was a problem hiding this comment.
We should make sure that this won't break any model
There was a problem hiding this comment.
I just followed what we did for the S2STransformerBeamSearch. The beam search algorithm expects to get the attn weights if provided. In our case, we have attn weights, so it makes sense to return them. One note: I think the return of attn from the decoder should be mandatory (and not optional).
I did some tests with and without attn in the beam search and did not see any changes. (might be related to how I initialized the beam search. I will investigate later tomorrow).
Avoid the inefficient conversion of input tensors to CPU NumPy arrays
TParcollet
left a comment
There was a problem hiding this comment.
Once the requested changes will be done and results added, the PR will be ready to be merger.
…peechbrain into whisper-finetuning
…eechbrain into whisper-finetuning
No description provided.