Why is correlation between my independent variables helping my linear regression model?Can we predict...
How to politely refuse in-office gym instructor for steroids and protein
Can you tell from a blurry photo if focus was too close or too far?
A Missing Symbol for This Logo
Is there a defined priority for pattern matching?
How much mayhem could I cause as a fish?
Citing paywalled articles accessed via illegal web sharing
Is using an 'empty' metaphor considered bad style?
Explanation of a regular pattern only occuring for prime numbers
Why is Agricola named as such?
Why avoid shared user accounts?
How do you funnel food off a cutting board?
Can I announce prefix 161.117.25.0/24 even though I don't have all of /24 IPs
Why did Democrats in the Senate oppose the Born-Alive Abortion Survivors Protection Act (2019 S.130)?
Why didn't Tom Riddle take the presence of Fawkes and the Sorting Hat as more of a threat?
How to access internet and run apt-get through a middle server?
What is the difference between rolling more dice versus fewer dice?
Changing the laptop's CPU. Should I reinstall Linux?
TikZ graph edges not drawn nicely
After checking in online, how do I know whether I need to go show my passport at airport check-in?
Boss asked me to sign a resignation paper without a date on it along with my new contract
What happens when I Twin Life Transference?
Why does photorec keep finding files after I have filled the disk free space as root?
Why was Lupin comfortable with saying Voldemort's name?
Does Skippy chunky peanut butter contain trans fat?
Why is correlation between my independent variables helping my linear regression model?
Can we predict correlation between independent variables based on dependent variablesWhy augmenting the training data with binary attributes works better for our dataset?Linear regression on probabilistic dataXGBoost: predict on only valuable featuresBest way to normalize datasets for a linear regression model?Choosing input variables for Linear Regression for higher accuracyPlease help to identify if there is meaning of linear variables separation by features in phase space?How to calculate and print unseen time series values for LSTM after train, valid and test dataNeed input on which features to drop in classification modelReducing Bias when trying to find Feature Importance using a Random Forest
$begingroup$
I am working with PUBG data and developing a linear regression model for the same ! Now there were three features in my original dataset, ridedistance, swimdistance, walkdistance. I combined the three with a new feature : distance covered which is the sum of the above mentioned three features. When putting it in a linearregression model, when I use the three features and the fourth one as well, I get a better score as compared to using the three featues only or using only the fourth feature. I have read that correlation between features when developing a model should not be there. But when all features (4 of them) having correlation are used to develop a model, the model has a better square (R-square). Why is this happening ?
python scikit-learn data linear-regression
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
$endgroup$
bumped to the homepage by Community♦ 5 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I am working with PUBG data and developing a linear regression model for the same ! Now there were three features in my original dataset, ridedistance, swimdistance, walkdistance. I combined the three with a new feature : distance covered which is the sum of the above mentioned three features. When putting it in a linearregression model, when I use the three features and the fourth one as well, I get a better score as compared to using the three featues only or using only the fourth feature. I have read that correlation between features when developing a model should not be there. But when all features (4 of them) having correlation are used to develop a model, the model has a better square (R-square). Why is this happening ?
python scikit-learn data linear-regression
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
$endgroup$
bumped to the homepage by Community♦ 5 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
$begingroup$
It would be better if you specify the model...is it a NN linear regression model? or simply a ML regression model? Also what are you trying to predict?
$endgroup$
– DuttaA
Jan 27 at 16:49
$begingroup$
@DuttaA It is a linear regression model from scikit-learn. There are 29 total columns (all continuous) and I am trying to predict a float number from 0 to 1 which can have any value upto 2 places of decimal.
$endgroup$
– Rishabh Sharma
Jan 27 at 16:50
$begingroup$
I don't know what sickit-learn is using, so you need to specify what exactly are you using...whether it is a NN based model or not..
$endgroup$
– DuttaA
Jan 27 at 16:52
$begingroup$
It;s not NN based model. It's a model based on gradient descent and regression much like y_predict = m1x1 + m2x2 + ... + c
$endgroup$
– Rishabh Sharma
Jan 27 at 16:58
add a comment |
$begingroup$
I am working with PUBG data and developing a linear regression model for the same ! Now there were three features in my original dataset, ridedistance, swimdistance, walkdistance. I combined the three with a new feature : distance covered which is the sum of the above mentioned three features. When putting it in a linearregression model, when I use the three features and the fourth one as well, I get a better score as compared to using the three featues only or using only the fourth feature. I have read that correlation between features when developing a model should not be there. But when all features (4 of them) having correlation are used to develop a model, the model has a better square (R-square). Why is this happening ?
python scikit-learn data linear-regression
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
$endgroup$
I am working with PUBG data and developing a linear regression model for the same ! Now there were three features in my original dataset, ridedistance, swimdistance, walkdistance. I combined the three with a new feature : distance covered which is the sum of the above mentioned three features. When putting it in a linearregression model, when I use the three features and the fourth one as well, I get a better score as compared to using the three featues only or using only the fourth feature. I have read that correlation between features when developing a model should not be there. But when all features (4 of them) having correlation are used to develop a model, the model has a better square (R-square). Why is this happening ?
python scikit-learn data linear-regression
python scikit-learn data linear-regression
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
asked Jan 27 at 12:30
Rishabh SharmaRishabh Sharma
554
554
bumped to the homepage by Community♦ 5 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 5 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
$begingroup$
It would be better if you specify the model...is it a NN linear regression model? or simply a ML regression model? Also what are you trying to predict?
$endgroup$
– DuttaA
Jan 27 at 16:49
$begingroup$
@DuttaA It is a linear regression model from scikit-learn. There are 29 total columns (all continuous) and I am trying to predict a float number from 0 to 1 which can have any value upto 2 places of decimal.
$endgroup$
– Rishabh Sharma
Jan 27 at 16:50
$begingroup$
I don't know what sickit-learn is using, so you need to specify what exactly are you using...whether it is a NN based model or not..
$endgroup$
– DuttaA
Jan 27 at 16:52
$begingroup$
It;s not NN based model. It's a model based on gradient descent and regression much like y_predict = m1x1 + m2x2 + ... + c
$endgroup$
– Rishabh Sharma
Jan 27 at 16:58
add a comment |
$begingroup$
It would be better if you specify the model...is it a NN linear regression model? or simply a ML regression model? Also what are you trying to predict?
$endgroup$
– DuttaA
Jan 27 at 16:49
$begingroup$
@DuttaA It is a linear regression model from scikit-learn. There are 29 total columns (all continuous) and I am trying to predict a float number from 0 to 1 which can have any value upto 2 places of decimal.
$endgroup$
– Rishabh Sharma
Jan 27 at 16:50
$begingroup$
I don't know what sickit-learn is using, so you need to specify what exactly are you using...whether it is a NN based model or not..
$endgroup$
– DuttaA
Jan 27 at 16:52
$begingroup$
It;s not NN based model. It's a model based on gradient descent and regression much like y_predict = m1x1 + m2x2 + ... + c
$endgroup$
– Rishabh Sharma
Jan 27 at 16:58
$begingroup$
It would be better if you specify the model...is it a NN linear regression model? or simply a ML regression model? Also what are you trying to predict?
$endgroup$
– DuttaA
Jan 27 at 16:49
$begingroup$
It would be better if you specify the model...is it a NN linear regression model? or simply a ML regression model? Also what are you trying to predict?
$endgroup$
– DuttaA
Jan 27 at 16:49
$begingroup$
@DuttaA It is a linear regression model from scikit-learn. There are 29 total columns (all continuous) and I am trying to predict a float number from 0 to 1 which can have any value upto 2 places of decimal.
$endgroup$
– Rishabh Sharma
Jan 27 at 16:50
$begingroup$
@DuttaA It is a linear regression model from scikit-learn. There are 29 total columns (all continuous) and I am trying to predict a float number from 0 to 1 which can have any value upto 2 places of decimal.
$endgroup$
– Rishabh Sharma
Jan 27 at 16:50
$begingroup$
I don't know what sickit-learn is using, so you need to specify what exactly are you using...whether it is a NN based model or not..
$endgroup$
– DuttaA
Jan 27 at 16:52
$begingroup$
I don't know what sickit-learn is using, so you need to specify what exactly are you using...whether it is a NN based model or not..
$endgroup$
– DuttaA
Jan 27 at 16:52
$begingroup$
It;s not NN based model. It's a model based on gradient descent and regression much like y_predict = m1x1 + m2x2 + ... + c
$endgroup$
– Rishabh Sharma
Jan 27 at 16:58
$begingroup$
It;s not NN based model. It's a model based on gradient descent and regression much like y_predict = m1x1 + m2x2 + ... + c
$endgroup$
– Rishabh Sharma
Jan 27 at 16:58
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
It seems that you are dealing with the problem of Multicollinearity. Multicollinearity happens when your predictors are correlated with other predictors in the model.
Moderate multicollinearity may not be problematic. However, severe multicollinearity is a problem because it can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. The result is that the coefficient estimates are unstable and difficult to interpret.
You can use adjusted R-squared to see if the new added variable is actually helping your model to better explain the variance.
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f44641%2fwhy-is-correlation-between-my-independent-variables-helping-my-linear-regression%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
It seems that you are dealing with the problem of Multicollinearity. Multicollinearity happens when your predictors are correlated with other predictors in the model.
Moderate multicollinearity may not be problematic. However, severe multicollinearity is a problem because it can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. The result is that the coefficient estimates are unstable and difficult to interpret.
You can use adjusted R-squared to see if the new added variable is actually helping your model to better explain the variance.
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
$endgroup$
add a comment |
$begingroup$
It seems that you are dealing with the problem of Multicollinearity. Multicollinearity happens when your predictors are correlated with other predictors in the model.
Moderate multicollinearity may not be problematic. However, severe multicollinearity is a problem because it can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. The result is that the coefficient estimates are unstable and difficult to interpret.
You can use adjusted R-squared to see if the new added variable is actually helping your model to better explain the variance.
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
$endgroup$
add a comment |
$begingroup$
It seems that you are dealing with the problem of Multicollinearity. Multicollinearity happens when your predictors are correlated with other predictors in the model.
Moderate multicollinearity may not be problematic. However, severe multicollinearity is a problem because it can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. The result is that the coefficient estimates are unstable and difficult to interpret.
You can use adjusted R-squared to see if the new added variable is actually helping your model to better explain the variance.
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
$endgroup$
It seems that you are dealing with the problem of Multicollinearity. Multicollinearity happens when your predictors are correlated with other predictors in the model.
Moderate multicollinearity may not be problematic. However, severe multicollinearity is a problem because it can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. The result is that the coefficient estimates are unstable and difficult to interpret.
You can use adjusted R-squared to see if the new added variable is actually helping your model to better explain the variance.
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
answered Jan 28 at 2:32
Amirhos ImaniAmirhos Imani
1334
1334
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f44641%2fwhy-is-correlation-between-my-independent-variables-helping-my-linear-regression%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
It would be better if you specify the model...is it a NN linear regression model? or simply a ML regression model? Also what are you trying to predict?
$endgroup$
– DuttaA
Jan 27 at 16:49
$begingroup$
@DuttaA It is a linear regression model from scikit-learn. There are 29 total columns (all continuous) and I am trying to predict a float number from 0 to 1 which can have any value upto 2 places of decimal.
$endgroup$
– Rishabh Sharma
Jan 27 at 16:50
$begingroup$
I don't know what sickit-learn is using, so you need to specify what exactly are you using...whether it is a NN based model or not..
$endgroup$
– DuttaA
Jan 27 at 16:52
$begingroup$
It;s not NN based model. It's a model based on gradient descent and regression much like y_predict = m1x1 + m2x2 + ... + c
$endgroup$
– Rishabh Sharma
Jan 27 at 16:58