Extract most important features (per Class) using mutual_info_classif
I'm using mutual_info_classif to determine the most important words for a binary text-classification task as:
mi_score = mutual_info_classif(X, y)
but the above gives an array of feature scores without reference to the corresponding classes
Is there a way to get the most important features per class using MI?
P.s., I've already tried Chi2 but it gives the same feature rank for both classes
scikit-learn text-classification
asked Nov 12 at 17:51
Stan
4952723
add a comment |
I'm using mutual_info_classif to determine the most important words for a binary text-classification task as:
mi_score = mutual_info_classif(X, y)
but the above gives an array of feature scores without reference to the corresponding classes
Is there a way to get the most important features per class using MI?
P.s., I've already tried Chi2 but it gives the same feature rank for both classes
scikit-learn text-classification
asked Nov 12 at 17:51
Stan
4952723
add a comment |
I'm using mutual_info_classif to determine the most important words for a binary text-classification task as:
mi_score = mutual_info_classif(X, y)
but the above gives an array of feature scores without reference to the corresponding classes
Is there a way to get the most important features per class using MI?
P.s., I've already tried Chi2 but it gives the same feature rank for both classes
scikit-learn text-classification
asked Nov 12 at 17:51
Stan
4952723
I'm using mutual_info_classif to determine the most important words for a binary text-classification task as:
mi_score = mutual_info_classif(X, y)
but the above gives an array of feature scores without reference to the corresponding classes
Is there a way to get the most important features per class using MI?
P.s., I've already tried Chi2 but it gives the same feature rank for both classes
scikit-learn text-classification
scikit-learn text-classification
asked Nov 12 at 17:51
Stan
4952723
asked Nov 12 at 17:51
Stan
4952723
asked Nov 12 at 17:51
Stan
4952723
asked Nov 12 at 17:51
Stan
4952723
asked Nov 12 at 17:51
Stan
4952723
4952723
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Mutual information is a measure of dependence between 2 variables. In your case, between each of the attribute variables and the "Class" variable. The mutual information will give a higher score, when the attribute variable creates a better split of the target variable. This means you only get one score that describes the strength between the attirubte and the class. The most important feature is the one that best distinguishes between all of the classes.
If you have a class with multiple labels (not a binary class), you can create a new class variable for each label by using dummy variables.
For example, assume your class name is CLASS, and it holds 3 different labels: "Red", "Green" and "Blue". Create 3 new target variables, the first will be called "Is_Red", and it will hold "Yes" if CLASS=="Red" or "No" Otherwise. In this manner you can see which attribute best distinguish between each specific instance of the class. You will have to run the mutual information per new class variable.
answered Nov 13 at 13:38
Roee Anuar
574514
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53267548%2fextract-most-important-features-per-class-using-mutual-info-classif%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Mutual information is a measure of dependence between 2 variables. In your case, between each of the attribute variables and the "Class" variable. The mutual information will give a higher score, when the attribute variable creates a better split of the target variable. This means you only get one score that describes the strength between the attirubte and the class. The most important feature is the one that best distinguishes between all of the classes.
If you have a class with multiple labels (not a binary class), you can create a new class variable for each label by using dummy variables.
For example, assume your class name is CLASS, and it holds 3 different labels: "Red", "Green" and "Blue". Create 3 new target variables, the first will be called "Is_Red", and it will hold "Yes" if CLASS=="Red" or "No" Otherwise. In this manner you can see which attribute best distinguish between each specific instance of the class. You will have to run the mutual information per new class variable.
answered Nov 13 at 13:38
Roee Anuar
574514
add a comment |
Mutual information is a measure of dependence between 2 variables. In your case, between each of the attribute variables and the "Class" variable. The mutual information will give a higher score, when the attribute variable creates a better split of the target variable. This means you only get one score that describes the strength between the attirubte and the class. The most important feature is the one that best distinguishes between all of the classes.
If you have a class with multiple labels (not a binary class), you can create a new class variable for each label by using dummy variables.
For example, assume your class name is CLASS, and it holds 3 different labels: "Red", "Green" and "Blue". Create 3 new target variables, the first will be called "Is_Red", and it will hold "Yes" if CLASS=="Red" or "No" Otherwise. In this manner you can see which attribute best distinguish between each specific instance of the class. You will have to run the mutual information per new class variable.
answered Nov 13 at 13:38
Roee Anuar
574514
add a comment |
Mutual information is a measure of dependence between 2 variables. In your case, between each of the attribute variables and the "Class" variable. The mutual information will give a higher score, when the attribute variable creates a better split of the target variable. This means you only get one score that describes the strength between the attirubte and the class. The most important feature is the one that best distinguishes between all of the classes.
If you have a class with multiple labels (not a binary class), you can create a new class variable for each label by using dummy variables.
For example, assume your class name is CLASS, and it holds 3 different labels: "Red", "Green" and "Blue". Create 3 new target variables, the first will be called "Is_Red", and it will hold "Yes" if CLASS=="Red" or "No" Otherwise. In this manner you can see which attribute best distinguish between each specific instance of the class. You will have to run the mutual information per new class variable.
answered Nov 13 at 13:38
Roee Anuar
574514
Mutual information is a measure of dependence between 2 variables. In your case, between each of the attribute variables and the "Class" variable. The mutual information will give a higher score, when the attribute variable creates a better split of the target variable. This means you only get one score that describes the strength between the attirubte and the class. The most important feature is the one that best distinguishes between all of the classes.
If you have a class with multiple labels (not a binary class), you can create a new class variable for each label by using dummy variables.
For example, assume your class name is CLASS, and it holds 3 different labels: "Red", "Green" and "Blue". Create 3 new target variables, the first will be called "Is_Red", and it will hold "Yes" if CLASS=="Red" or "No" Otherwise. In this manner you can see which attribute best distinguish between each specific instance of the class. You will have to run the mutual information per new class variable.
answered Nov 13 at 13:38
Roee Anuar
574514
answered Nov 13 at 13:38
Roee Anuar
574514
answered Nov 13 at 13:38
Roee Anuar
574514
answered Nov 13 at 13:38
Roee Anuar
574514
574514
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53267548%2fextract-most-important-features-per-class-using-mutual-info-classif%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
