dplyr group_by() leaves some variables
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
0
I have a dataframe:
> head(APD.dataframe)
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 912009913 9 912 0 1.00252 24.9681 0.000059
2 912009913 9 912 0 1.00452 29.9591 -0.002713
3 912009913 9 912 0 1.00537 34.9494 -0.018948
4 912009913 9 912 0 1.00560 44.9372 -0.023865
5 912009913 9 912 0 1.00683 49.9329 -0.032359
6 912009913 9 912 0 1.00690 54.9625 -0.039507
And I want to subtract some data points from it according to Serial_number:
library(dplyr)
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
Then I fit this data frame: summary(fit3_irr<- lmer(log(log(Amplification)) ~ poly(Voltage, 3) + (poly(Voltage, 3) | Serial_number), REML = FALSE, data = APD_irr))
Problem now is, that I want to make use of ggeffects() by using ggpredict(fit3_irr) I receive the following error:
ggeffect(fit3_irr)
Error: Unknown column `log(log(Amplification))`
In addition: Warning message:
'glue::collapse' is deprecated.
Use 'glue_collapse' instead.
See help("Deprecated") and help("glue-deprecated").
As far as I understand, this is due to the group_by from dplyr at the beginning. But I'm not able to remove it. Thought of using ungroup() but either I do not understand how it works or it does not work in general:
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
I use R 3.4.4
edit: part of the data frame:
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 608004648 6 608 1 111.997 379.980 0.386364
2 608004648 6 608 1 123.673 381.968 0.381323
3 608004648 6 608 1 137.701 383.979 0.411581
4 608004648 6 608 1 154.514 385.973 0.460648
5 608004648 6 608 1 175.331 387.980 0.506632
6 608004648 6 608 1 201.379 389.968 0.554607
7 608004649 6 608 1 118.753 378.080 0.960515
8 608004649 6 608 1 131.739 380.085 1.060070
9 608004649 6 608 1 147.294 382.082 1.195970
10 608004649 6 608 1 166.238 384.077 1.369470
11 608004649 6 608 1 189.841 386.074 1.576770
12 608004649 6 608 1 220.072 388.073 1.849820
...
r dplyr
asked Nov 16 '18 at 10:30
BenBen
434517
|
show 2 more comments
0
I have a dataframe:
> head(APD.dataframe)
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 912009913 9 912 0 1.00252 24.9681 0.000059
2 912009913 9 912 0 1.00452 29.9591 -0.002713
3 912009913 9 912 0 1.00537 34.9494 -0.018948
4 912009913 9 912 0 1.00560 44.9372 -0.023865
5 912009913 9 912 0 1.00683 49.9329 -0.032359
6 912009913 9 912 0 1.00690 54.9625 -0.039507
And I want to subtract some data points from it according to Serial_number:
library(dplyr)
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
Then I fit this data frame: summary(fit3_irr<- lmer(log(log(Amplification)) ~ poly(Voltage, 3) + (poly(Voltage, 3) | Serial_number), REML = FALSE, data = APD_irr))
Problem now is, that I want to make use of ggeffects() by using ggpredict(fit3_irr) I receive the following error:
ggeffect(fit3_irr)
Error: Unknown column `log(log(Amplification))`
In addition: Warning message:
'glue::collapse' is deprecated.
Use 'glue_collapse' instead.
See help("Deprecated") and help("glue-deprecated").
As far as I understand, this is due to the group_by from dplyr at the beginning. But I'm not able to remove it. Thought of using ungroup() but either I do not understand how it works or it does not work in general:
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
I use R 3.4.4
edit: part of the data frame:
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 608004648 6 608 1 111.997 379.980 0.386364
2 608004648 6 608 1 123.673 381.968 0.381323
3 608004648 6 608 1 137.701 383.979 0.411581
4 608004648 6 608 1 154.514 385.973 0.460648
5 608004648 6 608 1 175.331 387.980 0.506632
6 608004648 6 608 1 201.379 389.968 0.554607
7 608004649 6 608 1 118.753 378.080 0.960515
8 608004649 6 608 1 131.739 380.085 1.060070
9 608004649 6 608 1 147.294 382.082 1.195970
10 608004649 6 608 1 166.238 384.077 1.369470
11 608004649 6 608 1 189.841 386.074 1.576770
12 608004649 6 608 1 220.072 388.073 1.849820
...
r dplyr
asked Nov 16 '18 at 10:30
BenBen
434517
4
Can you provide a reproducible example usingdput()?
– wl1234
Nov 16 '18 at 10:32
Thank you! I added a part of it (hope that is enough? Otherwise I can add more). dput() did not help as the output was too much for the console.
– Ben
Nov 16 '18 at 12:29
1
There's a lot of code here and two printouts of the data, neither of which are easily reproducible. That makes the question hard to follow. To keep things both minimal and reproducible, you can find a sample of data that recreates the issue anddputjust that, and you can scale down the code to just what is necessary to help debug
– camille
Nov 16 '18 at 14:10
Also one thing that is confusing me and maybe tripping you up as well is switching between usingdplyrfunctions & syntax, but then jumping back into base R filtering. Maybe being more consistent will help you debug?
– camille
Nov 16 '18 at 14:11
1
I think a workaround would be transforming the dependent variable before you use it in your model.
– Daniel
Nov 30 '18 at 7:57
|
show 2 more comments
0
0
0
I have a dataframe:
> head(APD.dataframe)
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 912009913 9 912 0 1.00252 24.9681 0.000059
2 912009913 9 912 0 1.00452 29.9591 -0.002713
3 912009913 9 912 0 1.00537 34.9494 -0.018948
4 912009913 9 912 0 1.00560 44.9372 -0.023865
5 912009913 9 912 0 1.00683 49.9329 -0.032359
6 912009913 9 912 0 1.00690 54.9625 -0.039507
And I want to subtract some data points from it according to Serial_number:
library(dplyr)
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
Then I fit this data frame: summary(fit3_irr<- lmer(log(log(Amplification)) ~ poly(Voltage, 3) + (poly(Voltage, 3) | Serial_number), REML = FALSE, data = APD_irr))
Problem now is, that I want to make use of ggeffects() by using ggpredict(fit3_irr) I receive the following error:
ggeffect(fit3_irr)
Error: Unknown column `log(log(Amplification))`
In addition: Warning message:
'glue::collapse' is deprecated.
Use 'glue_collapse' instead.
See help("Deprecated") and help("glue-deprecated").
As far as I understand, this is due to the group_by from dplyr at the beginning. But I'm not able to remove it. Thought of using ungroup() but either I do not understand how it works or it does not work in general:
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
I use R 3.4.4
edit: part of the data frame:
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 608004648 6 608 1 111.997 379.980 0.386364
2 608004648 6 608 1 123.673 381.968 0.381323
3 608004648 6 608 1 137.701 383.979 0.411581
4 608004648 6 608 1 154.514 385.973 0.460648
5 608004648 6 608 1 175.331 387.980 0.506632
6 608004648 6 608 1 201.379 389.968 0.554607
7 608004649 6 608 1 118.753 378.080 0.960515
8 608004649 6 608 1 131.739 380.085 1.060070
9 608004649 6 608 1 147.294 382.082 1.195970
10 608004649 6 608 1 166.238 384.077 1.369470
11 608004649 6 608 1 189.841 386.074 1.576770
12 608004649 6 608 1 220.072 388.073 1.849820
...
r dplyr
asked Nov 16 '18 at 10:30
BenBen
434517
I have a dataframe:
> head(APD.dataframe)
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 912009913 9 912 0 1.00252 24.9681 0.000059
2 912009913 9 912 0 1.00452 29.9591 -0.002713
3 912009913 9 912 0 1.00537 34.9494 -0.018948
4 912009913 9 912 0 1.00560 44.9372 -0.023865
5 912009913 9 912 0 1.00683 49.9329 -0.032359
6 912009913 9 912 0 1.00690 54.9625 -0.039507
And I want to subtract some data points from it according to Serial_number:
library(dplyr)
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
Then I fit this data frame: summary(fit3_irr<- lmer(log(log(Amplification)) ~ poly(Voltage, 3) + (poly(Voltage, 3) | Serial_number), REML = FALSE, data = APD_irr))
Problem now is, that I want to make use of ggeffects() by using ggpredict(fit3_irr) I receive the following error:
ggeffect(fit3_irr)
Error: Unknown column `log(log(Amplification))`
In addition: Warning message:
'glue::collapse' is deprecated.
Use 'glue_collapse' instead.
See help("Deprecated") and help("glue-deprecated").
As far as I understand, this is due to the group_by from dplyr at the beginning. But I'm not able to remove it. Thought of using ungroup() but either I do not understand how it works or it does not work in general:
APDmin_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(n, Amplification) %>% ungroup()
APDmin_irr<- APDmin_irr[!APDmin_irr$Threshold == "No", ]
APDmax_irr<- APD.frame_irr %>% group_by(Serial_number, Threshold) %>% top_n(-n, Amplification) %>% ungroup()
APDmax_irr<- APDmax_irr[!APDmax_irr$Threshold == "Yes", ]
APD_irr<- rbind(APDmin_irr, APDmax_irr)
APD_irr<- with(APD_irr, APD_irr[order(Serial_number, Amplification),])
APD_irr$Threshold<- NULL
I use R 3.4.4
edit: part of the data frame:
Serial_number Lot Wafer Irradiated Amplification Voltage Dark_current
1 608004648 6 608 1 111.997 379.980 0.386364
2 608004648 6 608 1 123.673 381.968 0.381323
3 608004648 6 608 1 137.701 383.979 0.411581
4 608004648 6 608 1 154.514 385.973 0.460648
5 608004648 6 608 1 175.331 387.980 0.506632
6 608004648 6 608 1 201.379 389.968 0.554607
7 608004649 6 608 1 118.753 378.080 0.960515
8 608004649 6 608 1 131.739 380.085 1.060070
9 608004649 6 608 1 147.294 382.082 1.195970
10 608004649 6 608 1 166.238 384.077 1.369470
11 608004649 6 608 1 189.841 386.074 1.576770
12 608004649 6 608 1 220.072 388.073 1.849820
...
r dplyr
r dplyr
asked Nov 16 '18 at 10:30
BenBen
434517
asked Nov 16 '18 at 10:30
BenBen
434517
edited Nov 16 '18 at 12:28
Ben
asked Nov 16 '18 at 10:30
BenBen
434517
asked Nov 16 '18 at 10:30
BenBen
434517
asked Nov 16 '18 at 10:30
BenBen
434517
434517
4
Can you provide a reproducible example usingdput()?
– wl1234
Nov 16 '18 at 10:32
Thank you! I added a part of it (hope that is enough? Otherwise I can add more). dput() did not help as the output was too much for the console.
– Ben
Nov 16 '18 at 12:29
1
There's a lot of code here and two printouts of the data, neither of which are easily reproducible. That makes the question hard to follow. To keep things both minimal and reproducible, you can find a sample of data that recreates the issue anddputjust that, and you can scale down the code to just what is necessary to help debug
– camille
Nov 16 '18 at 14:10
Also one thing that is confusing me and maybe tripping you up as well is switching between usingdplyrfunctions & syntax, but then jumping back into base R filtering. Maybe being more consistent will help you debug?
– camille
Nov 16 '18 at 14:11
1
I think a workaround would be transforming the dependent variable before you use it in your model.
– Daniel
Nov 30 '18 at 7:57
|
show 2 more comments
4
Can you provide a reproducible example usingdput()?
– wl1234
Nov 16 '18 at 10:32
Thank you! I added a part of it (hope that is enough? Otherwise I can add more). dput() did not help as the output was too much for the console.
– Ben
Nov 16 '18 at 12:29
1
There's a lot of code here and two printouts of the data, neither of which are easily reproducible. That makes the question hard to follow. To keep things both minimal and reproducible, you can find a sample of data that recreates the issue anddputjust that, and you can scale down the code to just what is necessary to help debug
– camille
Nov 16 '18 at 14:10
Also one thing that is confusing me and maybe tripping you up as well is switching between usingdplyrfunctions & syntax, but then jumping back into base R filtering. Maybe being more consistent will help you debug?
– camille
Nov 16 '18 at 14:11
1
I think a workaround would be transforming the dependent variable before you use it in your model.
– Daniel
Nov 30 '18 at 7:57
4
4
Can you provide a reproducible example using
dput()?– wl1234
Nov 16 '18 at 10:32
Can you provide a reproducible example using
dput()?– wl1234
Nov 16 '18 at 10:32
Thank you! I added a part of it (hope that is enough? Otherwise I can add more). dput() did not help as the output was too much for the console.
– Ben
Nov 16 '18 at 12:29
Thank you! I added a part of it (hope that is enough? Otherwise I can add more). dput() did not help as the output was too much for the console.
– Ben
Nov 16 '18 at 12:29
1
1
There's a lot of code here and two printouts of the data, neither of which are easily reproducible. That makes the question hard to follow. To keep things both minimal and reproducible, you can find a sample of data that recreates the issue and
dput just that, and you can scale down the code to just what is necessary to help debug– camille
Nov 16 '18 at 14:10
There's a lot of code here and two printouts of the data, neither of which are easily reproducible. That makes the question hard to follow. To keep things both minimal and reproducible, you can find a sample of data that recreates the issue and
dput just that, and you can scale down the code to just what is necessary to help debug– camille
Nov 16 '18 at 14:10
Also one thing that is confusing me and maybe tripping you up as well is switching between using
dplyr functions & syntax, but then jumping back into base R filtering. Maybe being more consistent will help you debug?– camille
Nov 16 '18 at 14:11
Also one thing that is confusing me and maybe tripping you up as well is switching between using
dplyr functions & syntax, but then jumping back into base R filtering. Maybe being more consistent will help you debug?– camille
Nov 16 '18 at 14:11
1
1
I think a workaround would be transforming the dependent variable before you use it in your model.
– Daniel
Nov 30 '18 at 7:57
I think a workaround would be transforming the dependent variable before you use it in your model.
– Daniel
Nov 30 '18 at 7:57
|
show 2 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53335985%2fdplyr-group-by-leaves-some-variables%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53335985%2fdplyr-group-by-leaves-some-variables%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
4
Can you provide a reproducible example using
dput()?– wl1234
Nov 16 '18 at 10:32
Thank you! I added a part of it (hope that is enough? Otherwise I can add more). dput() did not help as the output was too much for the console.
– Ben
Nov 16 '18 at 12:29
1
There's a lot of code here and two printouts of the data, neither of which are easily reproducible. That makes the question hard to follow. To keep things both minimal and reproducible, you can find a sample of data that recreates the issue and
dputjust that, and you can scale down the code to just what is necessary to help debug– camille
Nov 16 '18 at 14:10
Also one thing that is confusing me and maybe tripping you up as well is switching between using
dplyrfunctions & syntax, but then jumping back into base R filtering. Maybe being more consistent will help you debug?– camille
Nov 16 '18 at 14:11
1
I think a workaround would be transforming the dependent variable before you use it in your model.
– Daniel
Nov 30 '18 at 7:57