Skip to content

Lagacy torch tensor constructor causes issue when using LM for scoring #6112

@advaitvd

Description

@advaitvd

Torch Version I'm using: 2.6.0+cu126

Issue Description:


Lagacy constructor expects the device type to be cpu thus causing issue when the AM decoder is on a gpu during inference. The above code should be updated to

torch.tensor(
  [self.sos] + max_hyp.yseq[1:],
  device=self.decoder.device,
  dtype=torch.long
)

Error Message:

Lagacy constructor for initializing Long Tensor needs to be updated.
Traceback (most recent call last):                                                                                                
  File "/opt/miniconda/lib/python3.9/site-packages/gradio/queueing.py", line 536, in process_events                               
    response = await route_utils.call_process_api(                                                                                
  File "/opt/miniconda/lib/python3.9/site-packages/gradio/route_utils.py", line 322, in call_process_api                          
    output = await app.get_blocks().process_api(                                                                                  
  File "/opt/miniconda/lib/python3.9/site-packages/gradio/blocks.py", line 1935, in process_api                                   
    result = await self.call_function(                                                                                            
  File "/opt/miniconda/lib/python3.9/site-packages/gradio/blocks.py", line 1520, in call_function                                 
    prediction = await anyio.to_thread.run_sync(  # type: ignore                                                                  
  File "/opt/miniconda/lib/python3.9/site-packages/anyio/to_thread.py", line 56, in run_sync                                      
    return await get_async_backend().run_sync_in_worker_thread(                                                                   
  File "/opt/miniconda/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread          
    return await future                                                                                                             File "/opt/miniconda/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 962, in run                                 
    result = context.run(func, *args)                            
  File "/opt/miniconda/lib/python3.9/site-packages/gradio/utils.py", line 826, in wrapper                                         
    response = f(*args, **kwargs)                                
  File "/app/workspace/ASR/gr_cln.py", line 52, in speech_to_text                                                                 
    n_best = speech2text(audio_data)                             
  File "/opt/miniconda/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context                     
    return func(*args, **kwargs)                                 
  File "/opt/miniconda/lib/python3.9/site-packages/espnet2/bin/asr_inference.py", line 546, in __call__                           
    results = self._decode_single_sample(enc[0])                                                                                  
  File "/opt/miniconda/lib/python3.9/site-packages/espnet2/bin/asr_inference.py", line 577, in _decode_single_sample              
    nbest_hyps = self.beam_search_transducer(enc)                                                                                 
  File "/opt/miniconda/lib/python3.9/site-packages/espnet2/asr/transducer/beam_search_transducer.py", line 180, in __call__
    nbest_hyps = self.search_algorithm(enc_out)                                                                                   
  File "/opt/miniconda/lib/python3.9/site-packages/espnet2/asr/transducer/beam_search_transducer.py", line 338, in default_beam_se
arch                            
    torch.LongTensor(                                            
RuntimeError: legacy constructor expects device type: cpu but device type: cuda was passed

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bugbug should be fixed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions