"遠程辦公時，您的團隊同時進行視頻會議的語音通話質量是否還有待提高呢？如果有這個疑惑，建議您看看這本書。本書圍繞視頻會議和遠場語音識別兩個熱門的領域展開，將音頻算法和工程實踐連成一體，從基礎理論到實踐方案，全面介紹業內主流的可商用的實時語音處理技術。語音算法開發中遇到的大部分問題均有涉及，並給出了比較專業的解法。"

●緒論······································································.1第1章 信號處理··············································.71.1 數字和模擬頻率··········································.71.2 離散傅裡葉變換···········································81.2.1 實數DFT ·····································.91.2.2 復數DFT ···································.101.2.3 負頻分量····································.101.2.4 DFT變換性質···························.101.3 FFT···························································.111.3.1 FFT 結果舉例····························.121.3.2 實信號FFT································.131.3.3 短時傅裡葉變換························.141.3.4 STFT語音窗函數選擇··············.141.4 重疊相加法和重疊保留法·························.161.4.1 OLA············································.171.4.2 OLS ············································.191.5 加權重疊相加法········································.211.5.1 WOLA 計算過程·······················.221.5.2 WOLA 窗函數選擇···················.221.6 濾波器組···················································.231.7 語音預加重····································.271.8 高斯分布···················································.271.8.1 單高斯分布································.271.8.2 多維高斯分布····························.291.9 HMM模型················································.311.10 卡爾曼濾波·············································.32本章小結·····························································.33參考文獻·····························································.33第2章 發音機理和器件·······························.342.1 語音的產生和接收········································.342.1.1 語音產生機理····························.342.1.2 發聲模型····································.362.1.3 發音單位····································.362.1.4 發音分類····································.372.1.5 聲音接收····································.372.1.6 聲音傳播····································.382.2 揚聲器·······················································.382.2.1 電學性能····································.382.2.2 聲學性能····································.392.2.3 底噪············································.402.2.4 頻響特性····································.412.2.5 THD+N POUT···························.412.2.6 電壓（功率）和失真················.422.3 麥克風·······················································.422.3.1 麥克風性能指標························.422.3.2 麥克風的選擇····························.432.4 結構設計····················································452.4.1 揚聲器相關音腔設計················.452.4.2 麥克風和揚聲器························.452.5 音頻設備···················································.462.5.1 聽音設備····································.462.5.2 聲場表現力································.472.5.3 發聲設備····································.482.5.4 消聲室測試································.482.6 聲學測試···················································.492.6.1 聲學音量····································.502.6.2 失真度THD·······························.502.6.3 頻響混疊····································.512.6.4 麥克風陣列一致性····················.532.6.5 AEC參考通路···························.542.6.6 揚聲器鏡頻································.562.6.7 揚聲器優選幅度下的THD·······.57本章小結·····························································.58參考文獻·····························································.58第3章 語音端點檢測····································.593.1 特征選取···················································.593.2 判決準則···················································.613.2.1 門限············································.613.2.2 統計模型法································.613.2.3 機器學習法································.623.3 VAD 實例·················································.633.3.1 高斯分布····································.633.3.2 算法流程····································.633.3.3 計算流程····································.683.4 語音/非語音幀的初始參數························.753.4.1 模型參數計算····························.753.4.2 高斯混合模型····························.763.4.3 EM算法·····································.76本章小結·····························································.78參考文獻·····························································.78第4章 單通道降噪········································.794.1 譜減法·······················································.794.1.1 譜減法原理································.794.1.2 譜減法實現································.814.1.3 音樂噪聲控制····························.834.1.4 濾波法········································.834.2 維納濾波···················································.844.3 子空間降噪···············································.864.4 WebRTC 單通道降噪實現······················.874.4.1 算法原理····································.874.4.2 算法初始化································.884.4.3 信噪比計算：ComputeSnr ·······.904.4.4 語音噪聲概率計算····················.914.4.5 特征選取····································.944.4.6 平坦度計算································.964.4.7 噪聲估計更新函數：UpdateNoiseEstimate···············.974.4.8 消除噪聲····································.984.4.9 信號合成····································.994.4.10 仿真結果··································.994.5 深度學習降噪········································.101本章小結···························································.104參考文獻···························································.105第5章 聲學回聲消除·································.1065.1 回聲消除原理·········································.1065.2 自適應濾波器·········································.1085.2.1 維納濾波器······························.1085.2.2 LMS算法································.1095.2.3 NLMS算法······························.1105.2.4 PBFDAF 算法··························.1115.3 WebRTC 回聲消除算法·······················.1135.3.1 延遲估計··································.1135.3.2 自適應濾波······························.1145.3.3 非線性處理（NLP）··············.1175.3.4 MATLAB代碼解讀················.1185.3.5 仿真實驗··································.1275.4 Speex 回聲消除算法·····························.1285.4.1 變步長計算······························.1295.4.2 雙線性濾波器及預處理··········.1305.4.3 MATLAB代碼解讀················.1325.4.4 算法流程示意圖······················.1415.4.5 仿真實驗··································.144本章小結···························································.146參考文獻···························································.146第6章 聲源定位··········································.1476.1 GCC算法·····················.1476.2 SRP-PHAT算法··································.1496.3 MUSIC算法···········································.1506.4 TOPS 算法·············································.1526.5 FRIDA算法············································.1546.6 後處理抗噪·············································.1556.6.1 統計方法··································.1556.6.2 卡爾曼方法······························.1566.6.3 聲源定位建模··························.1586.6.4 粒子濾波法······························.160本章小結···························································.160參考文獻···························································.161第7章 波束形成技術··································.1627.1 麥克風陣列·············································.1637.1.1 麥克風數量和間距··················.1637.1.2 空域混疊··································.1657.1.3 波束形成指標··························.1657.1.4 噪聲場······································.1667.1.5 聲輻射······································.1677.2 常見波束形成方法··································.1687.2.1 延遲和波束形成方法··············.1687.2.2 濾波和波束形成方法··············.1697.2.3 恆定寬度波束形成方法··········.1697.2.4 超分辨波束形成方法··············.1707.2.5 廣義旁瓣相消波束形成方法··.1717.2.6 最小方差信號無畸變響應波束形成方法················.1727.3 WebRTC 波束形成實例·······················.1747.3.1 編譯測試文件··························.1747.3.2 測試文件處理流程··················.1757.3.3 測試命令··································.1767.3.4 算法的基本思想······················.1767.3.5 測試源碼··································.1787.3.6 算法處理流程··························.1817.3.7 權重計算函數··························.1857.3.8 權重相乘操作··························.1867.4 後置濾波（ t-filtering） ·················.1877.4.1 MMSE後置濾波·····················.1897.4.2 Zelinski 後置濾波····················.1907.4.3 mccowan後置濾波·················.1917.4.4 STSA後置濾波·······················.192本章小結···························································.193參考文獻···························································.194第8章 盲源分離··········································.1968.1 基本概念及數學預備知識······················.1968.1.1 ICA基本概念··························.1968.1.2 梯度和很優化方法··················.1978.2 盲語音分離預處理――PCA··················.1998.3 頻域獨立成分分析法――FDICA··········.2008.3.1 頻域ICA··································.2008.3.2 去相關估計方法······················.2008.3.3 不確定性問題··························.2018.4 後置濾波處理··········································.2058.4.1 噪聲估計··································.2058.4.2 衰減因子計算··························.2068.5 GSC 與ICA聯合估計···························.2098.5.1 峭度··········································.2098.5.2 經典GSC·································.2108.5.3 動態權重向量估計··················.210本章小結···························································.212參考文獻···························································.213第9章 音效處理··········································.2149.1 聲道的分類·············································.2149.1.1 單聲道······································.2149.1.2 雙聲道······································.2159.1.3 立體聲······································.2159.1.4 多聲道······································.2159.1.5 全景聲······································.2169.2 後端音效處理··········································.217本章小結···························································.226參考文獻···························································.226第10章 語音編/解碼··································.22710.1 LPC 編碼·············································.23010.2 SILK編/解碼········································.23110.2.1 編碼參數································.23210.2.2 編碼器····································.23410.2.3 解碼器····································.23910.3 opus 編/解碼概覽································.23910.3.1 opus 解碼·······························.24210.3.2 opus 編碼·······························.24310.3.3 opus 語音/音樂檢測·············.24410.4 語音質量評估·······································.24710.4.1 主觀測試································.24810.4.2 客觀測試································.24810.4.3 無參考質量評估····················.249本章小結···························································.249參考文獻···························································.249第11章 語音網絡傳輸·······························.25111.1 擁塞控制···············································.25211.1.1 GoogleCC擁塞控制··············.25511.1.2 基於PCC的擁塞控制··········.26011.1.3 基於BBR 的擁塞控制··········.26411.2 NetEQ ·················································.26611.2.1 NetEQ原理····························.26611.2.2 抖動和收包····························.26811.2.3 NetEQ代碼框架····················.26911.2.4 延遲計算································.27211.2.5 DSP 處理·······························.27411.2.6 變速不變調····························.275本章小結···························································.277參考文獻···························································.277第12章 語音喚醒·······································.27812.1 語音喚醒技術簡介································.27812.2 特征提取···············································.27912.2.1 FBank ·····································.27912.2.2 MFCC·····································.28312.2.3 PCEN ·····································.28412.3 模型結構···············································.28412.3.1 DNN ·······································.28412.3.2 CNN ·······································.28612.3.3 CRNN·····································.28712.3.4 DSCNN ··································.28812.3.5 子帶CNN ······························.28912.3.6 Attention·································.29012.4 計算加速···············································.29212.4.1 硬件資源評估························.29212.4.2 加速方向································.294本章小結···························································.299參考文獻···························································.299第13章 語音識別·······································.30113.1 語音特征提取·······································.30313.1.1 MFCC特征····························.30413.1.2 PLP 特征································.30513.1.3 歸一化····································.30613.2 聲學模型···············································.30613.2.1 高斯混合模型························.30713.2.2 參數估計································.30713.2.3 隱馬爾科夫模型····················.30813.2.4 Baum-Welch法······················.30913.2.5 HMM識別器·························.30913.3 語言模型···············································.31013.3.1 N-gram語言模型··················.31113.3.2 加權有限狀態轉換機············.31213.4 YES 和NO識別實例···························31213.4.1 數據準備································.31213.4.2 數據預處理····························.31313.4.3 詞彙和發音詞典····················.31413.4.4 語言學模型····························.31513.4.5 特征提取································.31913.4.6 聲學模型訓練························.32013.4.7 解碼和測試····························.32113.5 Kaldi 中文語音識別······························32113.5.1 數據集準備····························.32113.5.2 聲學模型訓練························.32213.5.3 安裝portaudio ·······················.32213.5.4 在線識別································.32313.6 DeepSpeech 語音識別······················.32413.6.1 識別建模································.32513.6.2 網絡組成································.32513.6.3 模型訓練和部署····················.326本章小結···························································.330參考文獻···························································.330附錄A 本書涉及的專業術語··························.331

本書主要介紹基於互聯網場景的交互式實時語音處理流程，內容涉及智能語音助手、智能音箱、音/視頻會議等，具體包括實時語音信號處理、數字音效、網絡傳輸編/解碼和語音喚醒識別四部分。在闡述各部分的內容時，本書從基本概念和原理入手，將理論和實踐相結合，並細致分析了極具商業價值的實例，以幫助讀者了解相關算法在工程上是如何實現的。另外，為便於有興趣的讀者快速進行算法驗證並將其改進和應用到實際的項目中，作者也開源了書中算法的源碼。

葛世超等 著