heatmap not representing data accurately
Below is a sample of my data set, which is simply 14 columns and about 6500 rows, filled with 1's and 0's. Download my entire data set here.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
2 1 1 1 1 1 1 1 1 1 1 1 0 1 1
3 1 1 1 1 1 1 1 1 1 1 1 0 1 1
4 1 1 1 1 1 1 1 1 1 1 1 0 1 1
5 1 1 1 0 0 0 1 1 1 1 1 0 1 1
6 1 1 1 1 1 1 1 1 1 1 1 0 1 0
7 1 1 1 1 1 1 1 1 1 1 1 0 1 1
8 1 1 1 1 1 1 0 1 1 1 1 0 1 1
9 1 1 1 1 1 1 1 1 1 1 1 0 1 1
Column 8 has 13 zeros, but these are not represented in the heatmap.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
5011 1 1 1 0 0 0 1 0 0 1 1 0 1 1
5801 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5803 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5809 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5812 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5813 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5815 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5817 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5818 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5823 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5826 1 1 1 1 1 1 1 0 0 1 0 0 1 0
6435 1 1 1 0 0 0 0 0 0 1 1 0 1 1
6436 1 1 1 0 0 0 0 0 0 1 1 0 1 1
How can I edit my code to have these 13 zeros represented in the heatmap as a white line like the other columns?
heatmap code
library(RColorBrewer)
library(openxlsx)
# Read in dataset
df <- read.xlsx("data set.xlsx", sheet = 1)
# Set as integer matrix
m <- as.matrix(df[, -1])
#Create colors
greyblack <- c("grey", "black")
pal <- colorRampPalette(greyblack)(100)
# Create heatmap
heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
r heatmap
asked Nov 14 '18 at 19:11
m8_m8_
32112
add a comment |
Below is a sample of my data set, which is simply 14 columns and about 6500 rows, filled with 1's and 0's. Download my entire data set here.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
2 1 1 1 1 1 1 1 1 1 1 1 0 1 1
3 1 1 1 1 1 1 1 1 1 1 1 0 1 1
4 1 1 1 1 1 1 1 1 1 1 1 0 1 1
5 1 1 1 0 0 0 1 1 1 1 1 0 1 1
6 1 1 1 1 1 1 1 1 1 1 1 0 1 0
7 1 1 1 1 1 1 1 1 1 1 1 0 1 1
8 1 1 1 1 1 1 0 1 1 1 1 0 1 1
9 1 1 1 1 1 1 1 1 1 1 1 0 1 1
Column 8 has 13 zeros, but these are not represented in the heatmap.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
5011 1 1 1 0 0 0 1 0 0 1 1 0 1 1
5801 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5803 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5809 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5812 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5813 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5815 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5817 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5818 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5823 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5826 1 1 1 1 1 1 1 0 0 1 0 0 1 0
6435 1 1 1 0 0 0 0 0 0 1 1 0 1 1
6436 1 1 1 0 0 0 0 0 0 1 1 0 1 1
How can I edit my code to have these 13 zeros represented in the heatmap as a white line like the other columns?
heatmap code
library(RColorBrewer)
library(openxlsx)
# Read in dataset
df <- read.xlsx("data set.xlsx", sheet = 1)
# Set as integer matrix
m <- as.matrix(df[, -1])
#Create colors
greyblack <- c("grey", "black")
pal <- colorRampPalette(greyblack)(100)
# Create heatmap
heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
r heatmap
asked Nov 14 '18 at 19:11
m8_m8_
32112
add a comment |
Below is a sample of my data set, which is simply 14 columns and about 6500 rows, filled with 1's and 0's. Download my entire data set here.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
2 1 1 1 1 1 1 1 1 1 1 1 0 1 1
3 1 1 1 1 1 1 1 1 1 1 1 0 1 1
4 1 1 1 1 1 1 1 1 1 1 1 0 1 1
5 1 1 1 0 0 0 1 1 1 1 1 0 1 1
6 1 1 1 1 1 1 1 1 1 1 1 0 1 0
7 1 1 1 1 1 1 1 1 1 1 1 0 1 1
8 1 1 1 1 1 1 0 1 1 1 1 0 1 1
9 1 1 1 1 1 1 1 1 1 1 1 0 1 1
Column 8 has 13 zeros, but these are not represented in the heatmap.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
5011 1 1 1 0 0 0 1 0 0 1 1 0 1 1
5801 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5803 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5809 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5812 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5813 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5815 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5817 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5818 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5823 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5826 1 1 1 1 1 1 1 0 0 1 0 0 1 0
6435 1 1 1 0 0 0 0 0 0 1 1 0 1 1
6436 1 1 1 0 0 0 0 0 0 1 1 0 1 1
How can I edit my code to have these 13 zeros represented in the heatmap as a white line like the other columns?
heatmap code
library(RColorBrewer)
library(openxlsx)
# Read in dataset
df <- read.xlsx("data set.xlsx", sheet = 1)
# Set as integer matrix
m <- as.matrix(df[, -1])
#Create colors
greyblack <- c("grey", "black")
pal <- colorRampPalette(greyblack)(100)
# Create heatmap
heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
r heatmap
asked Nov 14 '18 at 19:11
m8_m8_
32112
Below is a sample of my data set, which is simply 14 columns and about 6500 rows, filled with 1's and 0's. Download my entire data set here.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
2 1 1 1 1 1 1 1 1 1 1 1 0 1 1
3 1 1 1 1 1 1 1 1 1 1 1 0 1 1
4 1 1 1 1 1 1 1 1 1 1 1 0 1 1
5 1 1 1 0 0 0 1 1 1 1 1 0 1 1
6 1 1 1 1 1 1 1 1 1 1 1 0 1 0
7 1 1 1 1 1 1 1 1 1 1 1 0 1 1
8 1 1 1 1 1 1 0 1 1 1 1 0 1 1
9 1 1 1 1 1 1 1 1 1 1 1 0 1 1
Column 8 has 13 zeros, but these are not represented in the heatmap.
Row C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
5011 1 1 1 0 0 0 1 0 0 1 1 0 1 1
5801 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5803 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5809 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5812 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5813 1 1 1 1 1 1 0 0 0 1 0 0 1 0
5815 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5817 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5818 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5823 1 1 1 1 1 1 1 0 0 1 0 0 1 0
5826 1 1 1 1 1 1 1 0 0 1 0 0 1 0
6435 1 1 1 0 0 0 0 0 0 1 1 0 1 1
6436 1 1 1 0 0 0 0 0 0 1 1 0 1 1
How can I edit my code to have these 13 zeros represented in the heatmap as a white line like the other columns?
heatmap code
library(RColorBrewer)
library(openxlsx)
# Read in dataset
df <- read.xlsx("data set.xlsx", sheet = 1)
# Set as integer matrix
m <- as.matrix(df[, -1])
#Create colors
greyblack <- c("grey", "black")
pal <- colorRampPalette(greyblack)(100)
# Create heatmap
heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
r heatmap
r heatmap
asked Nov 14 '18 at 19:11
m8_m8_
32112
asked Nov 14 '18 at 19:11
m8_m8_
32112
edited Nov 14 '18 at 19:23
m8_
asked Nov 14 '18 at 19:11
m8_m8_
32112
asked Nov 14 '18 at 19:11
m8_m8_
32112
asked Nov 14 '18 at 19:11
m8_m8_
32112
32112
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
This seems to be a problem of resolution. Each of these missing rows in col 8 are a single pixel wide. Column 1 also has zeroes that don't appear (rows 2811 and 3639), and the other columns have plenty of single-row (pixel) spaces of missing data that don't appear. When I cut the dataset down to a small area around one of the zeros in C8, they appear:
smalldf <- df[5009:5013,]
smallm <- as.matrix(smalldf[,-1])
heatmap(smallm, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
Image: When df is cut down to the area around one of the zeros on column 8, the white line shows up.
If you really want to show these 1-row-missing data in the heatmap, I would suggest replacing the adjacent 1s with zeroes. (ie, 1 zero becomes 1 zero above it and 1 zero below it.). A quick and dirty way to do that for one column:
> idx <- which(df$Column.8 == 0)
> df$Column.8[idx-2] <- 0
> df$Column.8[idx-1] <- 0
> df$Column.8[idx+1] <- 0
> df$Column.8[idx+2] <- 0
> m <- as.matrix(df[, -1])
> heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
This causes the lines to show up at least, although it appears as if there are only 2-3 discrete holes rather than 8 individual, tiny holes:
Heatmap with 'resampled' column 8
I'm not sure if this is a good way to do it, because I'm not sure what you're using this data/heatmap for. If it's just visualization of where the missing data is, then this should help. (And you may want to do the same thing for other columns)
answered Nov 15 '18 at 0:00
kaabrekaabre
12
thanks. However, I am receiving the following error:Error in $<-.data.frame(*tmp*, Column.8, value = numeric(0)) : replacement has 0 rows, data has 8310.
– m8_
Nov 15 '18 at 13:12
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53307217%2fheatmap-not-representing-data-accurately%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
This seems to be a problem of resolution. Each of these missing rows in col 8 are a single pixel wide. Column 1 also has zeroes that don't appear (rows 2811 and 3639), and the other columns have plenty of single-row (pixel) spaces of missing data that don't appear. When I cut the dataset down to a small area around one of the zeros in C8, they appear:
smalldf <- df[5009:5013,]
smallm <- as.matrix(smalldf[,-1])
heatmap(smallm, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
Image: When df is cut down to the area around one of the zeros on column 8, the white line shows up.
If you really want to show these 1-row-missing data in the heatmap, I would suggest replacing the adjacent 1s with zeroes. (ie, 1 zero becomes 1 zero above it and 1 zero below it.). A quick and dirty way to do that for one column:
> idx <- which(df$Column.8 == 0)
> df$Column.8[idx-2] <- 0
> df$Column.8[idx-1] <- 0
> df$Column.8[idx+1] <- 0
> df$Column.8[idx+2] <- 0
> m <- as.matrix(df[, -1])
> heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
This causes the lines to show up at least, although it appears as if there are only 2-3 discrete holes rather than 8 individual, tiny holes:
Heatmap with 'resampled' column 8
I'm not sure if this is a good way to do it, because I'm not sure what you're using this data/heatmap for. If it's just visualization of where the missing data is, then this should help. (And you may want to do the same thing for other columns)
answered Nov 15 '18 at 0:00
kaabrekaabre
12
thanks. However, I am receiving the following error:Error in $<-.data.frame(*tmp*, Column.8, value = numeric(0)) : replacement has 0 rows, data has 8310.
– m8_
Nov 15 '18 at 13:12
add a comment |
This seems to be a problem of resolution. Each of these missing rows in col 8 are a single pixel wide. Column 1 also has zeroes that don't appear (rows 2811 and 3639), and the other columns have plenty of single-row (pixel) spaces of missing data that don't appear. When I cut the dataset down to a small area around one of the zeros in C8, they appear:
smalldf <- df[5009:5013,]
smallm <- as.matrix(smalldf[,-1])
heatmap(smallm, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
Image: When df is cut down to the area around one of the zeros on column 8, the white line shows up.
If you really want to show these 1-row-missing data in the heatmap, I would suggest replacing the adjacent 1s with zeroes. (ie, 1 zero becomes 1 zero above it and 1 zero below it.). A quick and dirty way to do that for one column:
> idx <- which(df$Column.8 == 0)
> df$Column.8[idx-2] <- 0
> df$Column.8[idx-1] <- 0
> df$Column.8[idx+1] <- 0
> df$Column.8[idx+2] <- 0
> m <- as.matrix(df[, -1])
> heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
This causes the lines to show up at least, although it appears as if there are only 2-3 discrete holes rather than 8 individual, tiny holes:
Heatmap with 'resampled' column 8
I'm not sure if this is a good way to do it, because I'm not sure what you're using this data/heatmap for. If it's just visualization of where the missing data is, then this should help. (And you may want to do the same thing for other columns)
answered Nov 15 '18 at 0:00
kaabrekaabre
12
thanks. However, I am receiving the following error:Error in $<-.data.frame(*tmp*, Column.8, value = numeric(0)) : replacement has 0 rows, data has 8310.
– m8_
Nov 15 '18 at 13:12
add a comment |
This seems to be a problem of resolution. Each of these missing rows in col 8 are a single pixel wide. Column 1 also has zeroes that don't appear (rows 2811 and 3639), and the other columns have plenty of single-row (pixel) spaces of missing data that don't appear. When I cut the dataset down to a small area around one of the zeros in C8, they appear:
smalldf <- df[5009:5013,]
smallm <- as.matrix(smalldf[,-1])
heatmap(smallm, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
Image: When df is cut down to the area around one of the zeros on column 8, the white line shows up.
If you really want to show these 1-row-missing data in the heatmap, I would suggest replacing the adjacent 1s with zeroes. (ie, 1 zero becomes 1 zero above it and 1 zero below it.). A quick and dirty way to do that for one column:
> idx <- which(df$Column.8 == 0)
> df$Column.8[idx-2] <- 0
> df$Column.8[idx-1] <- 0
> df$Column.8[idx+1] <- 0
> df$Column.8[idx+2] <- 0
> m <- as.matrix(df[, -1])
> heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
This causes the lines to show up at least, although it appears as if there are only 2-3 discrete holes rather than 8 individual, tiny holes:
Heatmap with 'resampled' column 8
I'm not sure if this is a good way to do it, because I'm not sure what you're using this data/heatmap for. If it's just visualization of where the missing data is, then this should help. (And you may want to do the same thing for other columns)
answered Nov 15 '18 at 0:00
kaabrekaabre
12
This seems to be a problem of resolution. Each of these missing rows in col 8 are a single pixel wide. Column 1 also has zeroes that don't appear (rows 2811 and 3639), and the other columns have plenty of single-row (pixel) spaces of missing data that don't appear. When I cut the dataset down to a small area around one of the zeros in C8, they appear:
smalldf <- df[5009:5013,]
smallm <- as.matrix(smalldf[,-1])
heatmap(smallm, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
Image: When df is cut down to the area around one of the zeros on column 8, the white line shows up.
If you really want to show these 1-row-missing data in the heatmap, I would suggest replacing the adjacent 1s with zeroes. (ie, 1 zero becomes 1 zero above it and 1 zero below it.). A quick and dirty way to do that for one column:
> idx <- which(df$Column.8 == 0)
> df$Column.8[idx-2] <- 0
> df$Column.8[idx-1] <- 0
> df$Column.8[idx+1] <- 0
> df$Column.8[idx+2] <- 0
> m <- as.matrix(df[, -1])
> heatmap(m, Rowv = NA, Colv = NA, scale="none", col = pal, ylab="Records", main="Data Completeness")
This causes the lines to show up at least, although it appears as if there are only 2-3 discrete holes rather than 8 individual, tiny holes:
Heatmap with 'resampled' column 8
I'm not sure if this is a good way to do it, because I'm not sure what you're using this data/heatmap for. If it's just visualization of where the missing data is, then this should help. (And you may want to do the same thing for other columns)
answered Nov 15 '18 at 0:00
kaabrekaabre
12
answered Nov 15 '18 at 0:00
kaabrekaabre
12
answered Nov 15 '18 at 0:00
kaabrekaabre
12
answered Nov 15 '18 at 0:00
kaabrekaabre
12
12
thanks. However, I am receiving the following error:Error in $<-.data.frame(*tmp*, Column.8, value = numeric(0)) : replacement has 0 rows, data has 8310.
– m8_
Nov 15 '18 at 13:12
add a comment |
thanks. However, I am receiving the following error:Error in $<-.data.frame(*tmp*, Column.8, value = numeric(0)) : replacement has 0 rows, data has 8310.
– m8_
Nov 15 '18 at 13:12
thanks. However, I am receiving the following error:
Error in $<-.data.frame(*tmp*, Column.8, value = numeric(0)) : replacement has 0 rows, data has 8310.– m8_
Nov 15 '18 at 13:12
thanks. However, I am receiving the following error:
Error in $<-.data.frame(*tmp*, Column.8, value = numeric(0)) : replacement has 0 rows, data has 8310.– m8_
Nov 15 '18 at 13:12
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53307217%2fheatmap-not-representing-data-accurately%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
