Normal Equation of Computing Parameters Analytically
Normal Equation
Note: [8:00 to 8:44 - The design matrix X (in the bottom right side of the slide) given in the example should have elements x with subscript 1 and superscripts varying from 1 to m because for all m training sets there are only 2 features x0 and x1. 12:56 - The X matrix is m by (n+1) and NOT n by n. ]
Gradient descent gives one way of minimizing J. Let’s discuss a second way of doing so, this time performing the minimization explicitly and without resorting to an iterative algorithm. In the "Normal Equation" method, we will minimize J by explicitly taking its derivatives with respect to the θj ’s, and setting them to zero. This allows us to find the optimum theta without iteration. The normal equation formula is given below:
There is no need to do feature scaling with the normal equation.
The following is a comparison of gradient descent and the normal equation:
With the normal equation, computing the inversion has complexity So if we have a very large number of features, the normal equation will be slow. In practice, when n exceeds 10,000 it might be a good time to go from a normal solution to an iterative process.
Normal Equation Noninvertibility
When implementing the normal equation in octave we want to use the 'pinv' function rather than 'inv.' The 'pinv' function will give you a value of θ even if is not invertible.
If is noninvertible, the common causes might be having :
- Redundant features, where two features are very closely related (i.e. they are linearly dependent)
- Too many features (e.g. m ≤ n). In this case, delete some features or use "regularization" (to be explained in a later lesson).
Solutions to the above problems include deleting a feature that is linearly dependent with another or deleting one or more features when there are too many features.
最新文章
- cf 730i
- mysql 的简单优化
- DNS劫持解决方法
- 分享Kali Linux 2016.2第49周镜像文件
- Git自动部署
- HDU2222 Keywords Search(AC自动机模板)
- ionic安装拍照选照片插件
- SSAS父子层次结构的增强-UnaryOperatorColumn属性
- 你绝对想不到R文件找不到(cannot resolve symbol R)的原因
- bash启动时加载配置文件过程
- js 深拷贝和浅拷贝
- Vue2.x源码学习笔记-Vue源码调试
- RPC远程过程调用
- [db]mysql全量迁移db
- apache kylin总结
- Linux 进程中 Stop, Park, Freeze【转】
- Elastix 安装G729 G723语音编码
- ZUFE2486 Heap 2017-05-31 14:37 41人阅读 评论(0) 收藏
- 分词中的HMM
- c#winform中如何修改ListView控件每个单元格的颜色