[sohailshk]

Summary

A np.int16 array (for example from soundfile.read(..., dtype="int16"), PyAudio or sounddevice) was turned into an int16 tensor without scaling, whereas load_bytes, the kaldi_ark branch and funasr/bin/realtime_ws.py scale PCM16 by 1/32768. On main:

• offline ASR frontend (extract_fbank + WavFrontend, the path Paraformer/SenseVoice inference take): NotImplementedError: mean(): could not infer output dtype ... Got: Short, because waveform * (1 << 15) wraps in int16 (frontend-level run, no ASR weights);
• fsmn-vad: the int16 samples reach the frontend as unscaled floats (code reading: torch.cat with the float cache promotes them), 32768 times too loud, and the segments change;
• with fs other than 16 kHz: TypeError: Expected floating point type for waveform tensor, but received torch.int16 from the resampler.

int16 numpy arrays are now scaled like the float equivalent. Other dtypes, torch tensors and non-sound data_type are unchanged.

Type of change

• Bug fix

Validation

• python -m compileall funasr examples tests
• python -m pytest tests/test_load_audio_int16_array.py -q: 3 of 5 fail on main, 5 pass.
• fsmn-vad on example/vad_example.wav (70 s) read with soundfile.read(path, dtype="int16"): main returns 3 segments [[70, 26160], [26750, 37670], [38210, 70450]]; with the fix it returns the same 4 segments as the float32 array [[70, 26270], [26770, 37600], [38210, 59550], [59830, 70450]].
• int16 at 8 kHz (fs=8000): TypeError on main; with the fix the same segments as the float32 array at 8 kHz.
• extract_fbank + WavFrontend with an int16 array at 16 kHz: NotImplementedError on main, features computed with the fix.
• test_load_audio_bytes and test_pcm_input_format pass.

User impact

Raw PCM16 buffers (PyAudio, sounddevice, soundfile with dtype="int16") can be passed straight to generate() and give the same result as float input, at any fs.

Notes for reviewers

• Only np.int16 is scaled. torch int16 tensors, int32 and uint8 arrays behave as before (int32 has no single obvious scale; the kaldi_ark branch divides it by 32768).
• funasr_onnx / funasr_torch load_data pass arrays through unchanged as well (code reading); they are separate packages and not changed here.
• 请问有没有paraformer实时和vad实时的一体的gpu调用方法，以及vad录音输入问题 #1997 passed int16 sounddevice chunks to fsmn-vad and was closed without a code change; its exact symptom was not reproduced here.
• CPU only; fsmn-vad end to end, ASR frontend at feature level only.