[机器学习-从入门到入土] 基础知识
个人导航
知乎：https://www.zhihu.com/people/byzh_rc
CSDN：https://blog.csdn.net/qq_54636039
注：本文仅对所述内容做了框架性引导，具体细节可查询其余相关资料or源码
参考文章：各方资料
文章目录
- [机器学习-从入门到入土] 基础知识
- 个人导航
- 中英文
- 权重shape
- 损失函数/误差函数/代价函数/成本函数/J ( w ) J(w)J(w)
- 向量求导公式
- 矩阵求导公式
中英文
| 中文 | 英文 |
|---|---|
| 线性回归 | linearregression |
| 欠拟合 | underfit |
| 过拟合 | overfit |
| 代价函数 | cost function |
| 正则化 | Regularization |
| 随机梯度下降SGD | stochasticgradient descent |
| 方差 | variance |
| 先验 | prior |
| 后验 | posterior |
| — | — |
| 线性分类 | linear classification |
| 判别函数 | discriminantfunction |
| 决策面 | decision surface |
| 决策边界 | decisionboundary |
| 最小二乘法 | ordinary least squares |
| 感知器 | perceptron |
| 逻辑回归 | logisticregression |
| 均方误差MSE | mean-squareerror |
| 交叉熵损失 | cross-entropyloss |
| — | — |
| 神经网络 | neuralnetwork |
| 激活函数 | activationfunction |
| 前向传播 | forwardpropagation |
| 反向传播BP | backpropagation |
| 有限差分 | finitedifferences |
| 中心差分 | central differences |
| — | — |
| 计算学习理论 | computationallearning theory |
| 概率近似正确PAC | probably approximately correct |
| 样本复杂度 | sample complexity |
| 一致性 | consistent |
| 版本空间 | version space |
| 不可知学习 | agnosticlearning |
| VC维 | VCdimension |
| — | — |
| 经验误差 | empiricalerror |
| 泛化误差 | generalizationerror |
| 留出法 | hold-out |
| 交叉验证 | cross validation |
| 自助法 | bootstrap |
| 性能 | performance |
| 混淆矩阵 | confusionmatrix |
| 查准率 | precision |
| 召回率 / 查全率 | recall |
| 曲线 | curve |
| — | — |
| 相关特征 | relevant feature |
| 无关特征 | irrelevantfeature |
| 冗余特征 | redundantfeature |
| 子集搜索 | subsetsearch |
| 序列前向搜索SFS | sequential forward selection |
| 序列后向搜索SFS | sequential backward selection |
| 子集评价 | subsetevaluation |
| 信息增益 | gain |
| 过滤式 | filter |
| 包裹式 | wrapper |
| 嵌入式 | embedded |
| 字典学习 | dictionary learning |
| 稀疏表示 | sparserepresentation |
| — | — |
| 降维 | featurereduction |
| 主成分分析PCA | principal components analysis |
| 线性判别分析LDA | linear discriminant analysis |
| 本征维度 | intrinsicdimension |
| 概率PCA | probabilisticPCA |
| 核化PCA | kernel PCA |
| 自编码器 | auto-encoder |
| 流形学习 | manifoldlearning |
| 等度量特征映射isomap | isometric feature mapping |
| 局部线性嵌入LLE | locally linear embedding |
| 随机近邻嵌入SNE | stochastic neighbor embedding |
| 维度灾难 | curse ofdimensionality |
| 度量学习 | metriclearning |
| — | — |
| 概率图模型 | probabilisticgraphicalmodel |
| 贝叶斯网络 | Bayesiannetwork |
| 马尔科夫随机场 | Markovrandom field |
| 条件独立 | conditional Independence |
| 团块 | clique |
| 道德化 | moralization |
| — | — |
| 图像分类 | image classification |
| 目标检测 | objectdetection |
| 图像分割 | imagesegmentation |
| 不变性 | invariance |
| 同变性 | equivariance |
| 卷积 | convolution |
| 膨胀/空洞 | dilated |
| 通道 | channel |
| 感受野 | receptivefield |
| 下采样 | downsampling |
| 上采样 | upsampling |
| 归纳偏置 | inductive bias |
| — | — |
| 词嵌入 | word embedding |
| 词袋模型BOW | bag-of-words |
| 生成词向量 | Word2Vec |
| — | — |
| 生成式模型 | generativemodel |
| 判别式模型 | discriminativemodel |
| 自回归AR | autoregression |
| 变分自编码器 | variational autoencoder |
| 生成对抗网络 | generative adversarial network |
| 扩散模型 | diffusionmodel |
权重shape
常用符号:W j i , Θ j i W_{ji},\quad \Theta_{ji}Wji,Θji
反着写是为了方便乘法:
W j i W_{ji}Wji: (hidden, input+1) [加一是偏置]
x i x_ixi: (input+1,) [单个样本]
->a j = ∑ i = 0 i n p u t w j i x i , i = 1... h i d d e n a_j=\sum_{i=0}^{input}w_{ji}x_i,\quad i = 1 ... hiddenaj=∑i=0inputwjixi,i=1...hidden: (hidden, )
如果正着写就要转置w T w^TwT
损失函数/误差函数/代价函数/成本函数/J ( w ) J(w)J(w)
损失函数(Loss)：更偏向单样本误差，记作E EE
误差函数(error)：和损失函数的含义几乎等价，多用于回归任务的表述
代价函数 / 成本函数(Cost)：更偏向全体样本的平均 / 总误差，记作J ( w ) = 1 m ∑ i = 1 m E ( i ) J(w)=\frac{1}{m}\sum_{i=1}^mE^{(i)}J(w)=m1∑i=1mE(i)
向量求导公式
∂ a x T ∂ x = ∂ a T x ∂ x = a ∂ a x T b ∂ x = b a ∂ a T x b T ∂ x = a b \frac{\partial ax^T}{\partial x}=\frac{\partial a^Tx}{\partial x} = a \\ \frac{\partial ax^Tb}{\partial x} = ba \\ \frac{\partial a^Txb^T}{\partial x} = ab∂x∂axT=∂x∂aTx=a∂x∂axTb=ba∂x∂aTxbT=ab
矩阵求导公式
对函数f ( w ) = ( A w ) T ( A w ) = w T A T A w f(w) = (Aw)^T (Aw) = w^T A^T A wf(w)=(Aw)T(Aw)=wTATAw：
∂ f ∂ w = 2 A T A w \frac{\partial f}{\partial w} = 2 A^T A w∂w∂f=2ATAw
对函数g ( w ) = b T w g(w) = b^T wg(w)=bTw：
∂ g ∂ w = b \frac{\partial g}{\partial w} = b∂w∂g=b
