Capture data loading from fread into a shiny withProgress bar in R
I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.
I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.
Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:
withProgress(message = 'Loading Data', {
d <- fread("file_name.csv") # Pick a random (large) file to load
})
The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.
Update
I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.
I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.
Here is an example of how fread displays the read in progress:
fread Documentation
If you look up the documentation for fread this is what it says about the built in showProgess option:
TRUE displays progress on the console using r.
It is produced in freads C code where the very nice (but R level)
txtProgressBar and tkProgressBar are not easily available.
showProgress = getOption("datatable.showProgress") # default: TRUE
r shiny progress-bar fread
asked Nov 13 '18 at 19:18
BearBear
445114
add a comment |
I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.
I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.
Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:
withProgress(message = 'Loading Data', {
d <- fread("file_name.csv") # Pick a random (large) file to load
})
The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.
Update
I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.
I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.
Here is an example of how fread displays the read in progress:
fread Documentation
If you look up the documentation for fread this is what it says about the built in showProgess option:
TRUE displays progress on the console using r.
It is produced in freads C code where the very nice (but R level)
txtProgressBar and tkProgressBar are not easily available.
showProgress = getOption("datatable.showProgress") # default: TRUE
r shiny progress-bar fread
asked Nov 13 '18 at 19:18
BearBear
445114
add a comment |
I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.
I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.
Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:
withProgress(message = 'Loading Data', {
d <- fread("file_name.csv") # Pick a random (large) file to load
})
The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.
Update
I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.
I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.
Here is an example of how fread displays the read in progress:
fread Documentation
If you look up the documentation for fread this is what it says about the built in showProgess option:
TRUE displays progress on the console using r.
It is produced in freads C code where the very nice (but R level)
txtProgressBar and tkProgressBar are not easily available.
showProgress = getOption("datatable.showProgress") # default: TRUE
r shiny progress-bar fread
asked Nov 13 '18 at 19:18
BearBear
445114
I am loading in a large data file using fread which is from: library(data.table). fread by default shows the status of how long it is taking to read in the file but it only prints to the console.
I would like to capture the loading time of data and display it in real time in the shiny application so the user understands why there is a delay and roughly how long the data read is taking. I would like to use withProgress since this produces a loading bar and you can display text defining what the program is doing.
Typically, withProgress is incremented by a numeric scale using incProgress, however, I am looking for a way to do something like this:
withProgress(message = 'Loading Data', {
d <- fread("file_name.csv") # Pick a random (large) file to load
})
The problem with the above code is that the load bar isn't being incremented (I don't think that withProgress detects the loading by fread automatically). The load bar shows up and only increments a small amount but it is not correctly capturing the loading time from fread.
Update
I was able to find this similar question, however, the only answer given to this question uses the loading bar to indicate the progress of loading many files (Loaded 2 of n files). My problem is trying to show the progression of reading in a single large file.
I know fread knows how many rows a file contains after the initial scan because in the console it does show the progress. I just need to know how to tap into that to use it in the progress bar.
Here is an example of how fread displays the read in progress:
fread Documentation
If you look up the documentation for fread this is what it says about the built in showProgess option:
TRUE displays progress on the console using r.
It is produced in freads C code where the very nice (but R level)
txtProgressBar and tkProgressBar are not easily available.
showProgress = getOption("datatable.showProgress") # default: TRUE
r shiny progress-bar fread
r shiny progress-bar fread
asked Nov 13 '18 at 19:18
BearBear
445114
asked Nov 13 '18 at 19:18
BearBear
445114
edited Dec 4 '18 at 22:26
Bear
asked Nov 13 '18 at 19:18
BearBear
445114
asked Nov 13 '18 at 19:18
BearBear
445114
asked Nov 13 '18 at 19:18
BearBear
445114
445114
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53288073%2fcapture-data-loading-from-fread-into-a-shiny-withprogress-bar-in-r%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53288073%2fcapture-data-loading-from-fread-into-a-shiny-withprogress-bar-in-r%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
