My Coding >
Programming language >
Python >
Scikitlearn >
Ridge regression for approximate solution of linear equation
Ridge regression for approximate solution of linear equationIn previous example, I’ve showed you how to ise linear regression to find relation between two data sets. In this example, I will show you a bit more complicated case, where we have two sets of input data and and one output, but furthermore, two sets of input data are correlated with pretty high coefficient of correlation (near 1 by module). In this case we have unlimited number of solutions and we need to chose the best one. Task for ridge regressionWe have some kind of data relation Y = W_{1}*X_{1} + W_{2}*X_{2} + W_{0} with correlation between X_{1} and X_{2} Here is “real” data
As it is very easy to see, X_{1} and X_{2} have correlation = 1 and we have unlimited number of solutions. Traditional way of solvingSpeaking mathematically, we need to find the minima of the function L(W_{2}, W_{1}, W_{0}) = Σ(W_{2}*x_{2i} + W_{1}*x_{1i} + W_{0}  y_{i})^{2} To find the minima, we need to solve two equations in partial derivatives ∂L/∂W_{2} = 0; ∂L/∂W_{1} = 0; ∂L/∂W_{0} = 0 but this system have unlimited number of solution and we need to make some mechanism to choose between them. We can introduce a requirement, what coefficient W_{2}, W_{1} and W_{0}, should be minimal by its absolute value. For doing this we will introduce new function: R = L(W_{2}, W_{1}, W_{0}) + C*(W^{2}_{2} + W^{2}_{1} + W^{2}_{0}) or R = Σ(W_{2}*x_{2i} + W_{1}*x_{1i} + W_{0}  y_{i})^{2} + C*(W^{2}_{2} + W^{2}_{1} + W^{2}_{0}) And we will need to find minima of this new function. It is easy to see, that if we will approximate C to 0, we will have our previous equation. Therefore it is a good step to use C =1 Now we need to solve system of three equations
Solving ridge regression with scikitlearnThis problem already implemented in scikitlearn Python library.
So we need to take coefficient if the correct order: W_{0} = 0.17; W_{1} = 0.62; W_{2} = 0.10

Last 10 artitles
9 popular artitles


© 2020 MyCoding.uk My blog about coding and further learning. This blog was writen with pure Perl and frontend output was performed with TemplateToolkit. 