●序前言常用符號表第一部分 機器學習基礎第1章 緒論31.1人工智能...............................41.1.1人工智能的發展歷史....................51.1.2人工智能的流派.......................71.2機器學習...............................71.3表示學習...............................81.3.1局部表示和分布式表示...................91.3.2表示學習...........................111.4深度學習...............................111.4.1端到端學習..........................121.5神經網絡...............................131.5.1人腦神經網絡........................131.5.2人工神經網絡........................141.5.3神經網絡的發展歷史....................151.6本書的知識體繫...........................171.7常用的深度學習框架.........................181.8總結和深入閱讀...........................20第2章 機器學習概述232.1基本概念...............................242.2機器學習的三個基本要素......................262.2.1模型..............................262.2.2學習準則...........................272.2.3優化算法...........................302.3機器學習的簡單示例——線性回歸.................332.3.1參數學習...........................342.4偏差-方差分解............................382.5機器學習算法的類型.........................412.6數據的特征表示...........................432.6.1傳統的特征學習.......................442.6.2深度學習方法........................462.7評價指標...............................462.8理論和定理..............................492.8.1PAC學習理論........................492.8.2沒有免費午餐定理......................502.8.3奧卡姆剃刀原理.......................502.8.4丑小鴨定理..........................512.8.5歸納偏置...........................512.9總結和深入閱讀...........................51第3章 線性模型3.1線性判別函數和決策邊界......................563.1.1二分類............................563.1.2多分類............................583.2Logistic回歸.............................593.2.1參數學習...........................603.3Softmax回歸.............................613.3.1參數學習...........................623.4感知器.................................643.4.1參數學習...........................643.4.2感知器的收斂性.......................663.4.3參數平均感知器.......................673.4.4擴展到多分類........................693.5支持向量機..............................713.5.1參數學習...........................733.5.2核函數............................743.5.3軟間隔............................743.6損失函數對比.............................753.7總結和深入閱讀...........................76第二部分 基礎模型第4章 前饋神經網絡814..................................824.1.1Sigmoid型函數.......................834.1.2ReLU函數..........................864.1.3Swish函數..........................884.1.4GELU函數..........................894.1.5Maxo.........................894.2網絡結構...............................904.2.1前饋網絡...........................904.2.2記憶網絡...........................904.2.3圖網絡............................904.3前饋神經網絡.............................914.3.1通用近似定理........................934.3.2應用到機器學習.......................944.3.3參數學習...........................954.4反向傳播算法.............................954.5自動梯度計算.............................984.5.1數值微分...........................994.5.2符號微分...........................994.5.3自動微分...........................1004.6優化問題...............................1034.6.1非凸優化問題........................1034.6.2梯度消失問題........................1044.7總結和深入閱讀...........................104第5章 卷積神經網絡1095.1卷積..................................1105.1.1卷積的定義..........................1105.1.2互相關............................1125.1.3卷積的變種..........................1135.1.4卷積的數學性質.......................1145.2卷積神經網絡.............................1155.2.1用卷積來代替全連接....................1155.2.2卷積層............................1165.2.3彙聚層............................1185.2.4卷積網絡的整體結構....................1195.3參數學習...............................1205.3.1卷積神經網絡的反向傳播算法...............1205.4幾種典型的卷積神經網絡......................1215.4.1LeNet-5............................1225.4.2AlexNet...........................1235.4.3Inception網絡........................1255.4.4殘差網絡...........................1265.5其他卷積方式.............................1275.5.1轉置卷積...........................1275.5.2空洞卷積...........................1295.6總結和深入閱讀...........................130第6章 循環神經網絡1336.1給網絡增加記憶能力.........................1346.1.1延時神經網絡........................1346.1.2有外部輸入的非線性自回歸模型..............1346.1.3循環神經網絡........................1356.2簡單循環網絡.............................1356.2.1循環神經網絡的計算能力..................1366.3應用到機器學習...........................1386.3.1序列到類別模式.......................1386.3.2同步的序列到序列模式...................1396.3.3異步的序列到序列模式...................1396.4參數學習...............................1406.4.1隨時間反向傳播算法....................1416.4.2實時循環學習算法......................1426.5長程依賴問題.............................1436.5.1改進方案...........................1446.6基於門控的循環神經網絡......................1456.6.1長短期記憶網絡.......................1456.6.2LSTM網絡的各種變體...................1476.6.3門控網絡......................1486.7深層循環神經網絡..........................1496.7.1堆疊循環神經網絡......................1506.7.2雙向循環神經網絡......................1506.8擴展到圖結構.............................1516.8.1遞歸神經網絡........................1516.8.2圖神經網絡..........................1526.9總結和深入閱讀...........................153第7章 網絡優化與正則化1577.1網絡優化...............................1577.1.1網絡結構多樣性.......................1587.1.2高維變量的非凸優化....................1587.1.3神經網絡優化的改善方法..................1607.2優化算法...............................1607.2.1小批量梯度下降.......................1607.2.2批量大小選擇........................1617.2.3學習率調整..........................1627.2.4梯度估計修正........................1677.2.5優化算法小結........................1707.3參數初始化..............................1717.3.1基於固定方差的參數初始化.................1727.3.2基於方差縮放的參數初始化.................1737.3.3正交初始化..........................1757.4數據預處理..............................1767.5逐層歸一化..............................1787.5.1批量歸一化..........................1797.5.2層歸一化...........................1817.5.3權重歸一化..........................1827.5.4局部響應歸一化.......................1827.6超參數優化..............................1837.6.1網格搜索...........................1837.6.2隨機搜索...........................1847.6.3貝葉斯優化..........................1847.6.4動態資源分配........................1857.6.5神經架構搜索........................1867.7網絡正則化..............................1867.7.1?1和?2正則化........................1877.7.2權重衰減...........................1887.7.3提前停止...........................1887.7.4丟棄法............................1897.7.5數據增強...........................1917.7.6標簽平滑...........................1917.8總結和深入閱讀...........................192第8章 注意力機制與外部記憶1978.1認知神經學中的注意力.......................1988.2注意力機制..............................1998.2.1注意力機制的變體......................2018.3自注意力模型.............................2038.4人腦中的記憶.............................2058.5記憶增強神經網絡..........................2078.5.1端到端記憶網絡.......................2088.5.2神經圖靈機..........................2108.6基於神經動力學的聯想記憶.....................2118.6.1Hopfiel網絡........................2128.6.2使用聯想記憶增加網絡容量.................2158.7總結和深入閱讀...........................215第9章 無監督學習2199.1無監督特征學習...........................2209.1.1主成分分析..........................2209.1.2稀疏編碼...........................2229.1.3自編碼器...........................2249.1.4稀疏自編碼器........................2259.1.5堆疊自編碼器........................2269.1.6降噪自編碼器........................2269.2概率密度估計.............................2279.2.1參數密度估計........................2279.2.2非參數密度估計.......................2299.3總結和深入閱讀...........................232第10章 模型獨立的學習方式23510.1集成學習...............................23510.1.1AdaBoost算法........................23710.2自訓練和協同訓練..........................24010.2.1自訓練............................24010.2.2協同訓練...........................24010.3多任務學習..............................24210.4遷移學習...............................24510.4.1歸納遷移學習........................24610.4.2轉導遷移學習........................24710.5終身學習...............................2491學習.................................25210.6.1基於優學習....................25310.6.2模型學習......................25410.7總結和深入閱讀...........................255第三部分 進階模型第11章 概率圖模型26111.1模型表示...............................26211.1.1有向圖模型..........................26311.1.2常見的有向圖模型......................26411.1.3無向圖模型..........................26711.1.4無向圖模型的概率分解...................26711.1.5常見的無向圖模型......................26911.1.6有向圖和無向圖之間的轉換.................27011.2學習..................................27111.2.1不含隱變量的參數估計...................27111.2.2含隱變量的參數估計....................27311.3推斷..................................27911.3.1準確推斷...........................27911.3.2近似推斷...........................28211.4變分推斷...............................28311.5基於采樣法的近似推斷.......................28511.5.1采樣法............................28511.5.2拒絕采樣...........................28711.5.3重要性采樣..........................28811.5.4馬爾可夫鏈蒙特卡羅方法..................28911.6總結和深入閱讀...........................292第12章 深度信念網絡29712.1玻爾茲曼機..............................29712.1.1生成模型...........................29912.1.2能量最小化與模擬退火...................30112.1.3參數學習...........................30212.2受限玻爾茲曼機...........................30412.2.1生成模型...........................30512.2.2參數學習...........................30712.2.3受限玻爾茲曼機的類型...................30812.3深度信念網絡.............................30912.3.1生成模型...........................31012.3.2參數學習...........................31012.4總結和深入閱讀...........................313第13章 深度生成模型31713.1概率生成模型.............................31813.1.1密度估計...........................31813.1.2生成樣本...........................31913.1.3應用於監督學習.......................31913.2變分自編碼器.............................31913.2.1含隱變量的生成模型....................31913.2.2推斷網絡...........................32113.2.3生成網絡...........................32313.2.4模型彙總...........................32313.2.5再參數化...........................32513.2.6訓練..............................32513.3生成對抗網絡.............................32713.3.1顯式密度模型和隱式密度模型...............32713.3.2網絡分解...........................32713.3.3訓練..............................32913.3.4一個生成對抗網絡的具體實現：DCGAN..........33013.3.5模型分析...........................33013.3.6改進模型...........................33313.4總結和深入閱讀...........................336第14章 深度強化學習33914.1強化學習問題.............................34014.1.1典型例子...........................34014.1.2強化學習定義........................34014.1.3馬爾可夫決策過程......................34114.1.4強化學習的目標函數....................34314.1.5值函數............................34414.1.6深度強化學習........................34514.2基於值函數的學習方法.......................34614.2.1動態規劃算法........................34614.2.2蒙特卡羅方法........................34914.2.3時序差分學習方法......................35014.2.4深度Q網絡..........................35314.3基於策略函數的學習方法......................35414.3.1REINFORCE算法......................35614.3.2帶基準線的REINFORCE算法...............35614.4演員-評論員算法...........................35814.5總結和深入閱讀...........................360第15章 序列生成模型36515.1序列概率模型.............................36615.1.1序列生成...........................36715統計模型.............................36815.3深度序列模型.............................37015.3.1模型結構...........................37015.3.2參數學習...........................37315.4評價方法...............................37315.4.1困惑度............................37315.4.2BLEU算法..........................37415.4.3ROUGE算法.........................37515.5序列生成模型中的學習問題.....................37515.5.1曝光偏差問題........................37615.5.2訓練目標不一致問題....................37715.5.3計算效率問題........................37715.6序列到序列模型...........................38515.6.1基於循環神經網絡的序列到序列模型...........38615.6.2基於注意力的序列到序列模型...............38715.6.3基於自注意力的序列到序列模型..............38815.7總結和深入閱讀...........................390附錄數學基礎 393附錄A 線性代數 394附錄B 微積分 404附錄C 數學優化 413附錄D 概率論 420附錄E 信息論 433索引 439

本書是深度學習領域的入門教材，繫統地整理了深度學習的知識體繫，由淺入深地闡述了深度學習的基礎知識、主要模型以及前沿研究熱點，使得讀者能有效地掌握深度學習的相關知識，並具備以深度學習技術來處理和解決大數據問題的能力。全書共 15章，分為三個部分。第一部分為機器學習基礎：第 1章是緒論，概要介紹人工智能、機器學習、深度學習；第 2～3章介紹機器學習的基礎知識。第二部分是基礎模型：第 4～6章分別講述三種主要的神經網絡模型：前饋神經網絡、卷積神經網絡和循環神經網絡；第 7章介紹神經網絡的優化與正則化方法；第 8章介紹神經網絡中的注意力機制和外部記憶；第 9章簡要介紹一些無監督學習方法；第 10章介紹一些模型獨立的機器學習方法，包括集成學習、自訓練、協同訓練、多任務學習、遷移學習、終身學習等。第三部分是進階模型：第 11章介紹概率圖模型的基本概念；第 12章介紹兩種早期的深度學習模型—等