How to reduce the Querying time for larger size data in MongoDB
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
I am trying to querying the collection in MongoDB which matches more than 10000 data for the query. Even though I have used index, the querying time exceeds 25 seconds.
For example, I am having a table People with field name, age.
I need to fetch the People data whose age is 25, if query finds the matched objects is 10000, then it takes time to fetch the whole data.
I have created index like db.people.createIndex({"age":1})
Here, how can I reduce the querying time
node.js json mongodb nosql mongodb-query
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
add a comment |
I am trying to querying the collection in MongoDB which matches more than 10000 data for the query. Even though I have used index, the querying time exceeds 25 seconds.
For example, I am having a table People with field name, age.
I need to fetch the People data whose age is 25, if query finds the matched objects is 10000, then it takes time to fetch the whole data.
I have created index like db.people.createIndex({"age":1})
Here, how can I reduce the querying time
node.js json mongodb nosql mongodb-query
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
add a comment |
I am trying to querying the collection in MongoDB which matches more than 10000 data for the query. Even though I have used index, the querying time exceeds 25 seconds.
For example, I am having a table People with field name, age.
I need to fetch the People data whose age is 25, if query finds the matched objects is 10000, then it takes time to fetch the whole data.
I have created index like db.people.createIndex({"age":1})
Here, how can I reduce the querying time
node.js json mongodb nosql mongodb-query
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
I am trying to querying the collection in MongoDB which matches more than 10000 data for the query. Even though I have used index, the querying time exceeds 25 seconds.
For example, I am having a table People with field name, age.
I need to fetch the People data whose age is 25, if query finds the matched objects is 10000, then it takes time to fetch the whole data.
I have created index like db.people.createIndex({"age":1})
Here, how can I reduce the querying time
node.js json mongodb nosql mongodb-query
node.js json mongodb nosql mongodb-query
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
asked Nov 16 '18 at 12:48
AravindhAravindh
7610
7610
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
run db.collection.find().explain() and make sure that your index is in fact used. Make sure that you do not have COLLSCANs there https://docs.mongodb.com/manual/reference/explain-results/.
if your documents have some/many large attributes and you need only some attributes try to request only them (e.g. only _id or _id and name). Less data transferred gives higher speed.
if your db does not fit in memory, make it fit in memory. Once the database does not fit the performance will be much worse.
if you are not running on a sharded cluster, create one based on a reasonable sharding key. Age may not be a good one because than all age=25 documents will end up on one node. Even if you have one computer with multiple CPUs it still may work better for you (if you have enough memory for that). It may even work the other way around. If you have a sharded cluster on one computer and your replicas do not fit in the memory, it may be better to use just one node.
answered Nov 16 '18 at 13:00
petrchpetrch
32627
I checked with db.collection.find().explain(), and I am sure that my indexing is working I used only the required fields i.e., upto 7 fields and I am using shared cluster
– Aravindh
Nov 16 '18 at 13:35
good, what about the rest of the list? How large is your database? does it fit in memory and how large is your mongodb cluster?
– petrch
Nov 16 '18 at 13:36
2.5 GB of total used space is 1.75GB
– Aravindh
Nov 16 '18 at 13:39
can you provide the find command or the pipeline query and output of the explain for that same command?
– petrch
Nov 16 '18 at 14:40
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53338251%2fhow-to-reduce-the-querying-time-for-larger-size-data-in-mongodb%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
run db.collection.find().explain() and make sure that your index is in fact used. Make sure that you do not have COLLSCANs there https://docs.mongodb.com/manual/reference/explain-results/.
if your documents have some/many large attributes and you need only some attributes try to request only them (e.g. only _id or _id and name). Less data transferred gives higher speed.
if your db does not fit in memory, make it fit in memory. Once the database does not fit the performance will be much worse.
if you are not running on a sharded cluster, create one based on a reasonable sharding key. Age may not be a good one because than all age=25 documents will end up on one node. Even if you have one computer with multiple CPUs it still may work better for you (if you have enough memory for that). It may even work the other way around. If you have a sharded cluster on one computer and your replicas do not fit in the memory, it may be better to use just one node.
answered Nov 16 '18 at 13:00
petrchpetrch
32627
I checked with db.collection.find().explain(), and I am sure that my indexing is working I used only the required fields i.e., upto 7 fields and I am using shared cluster
– Aravindh
Nov 16 '18 at 13:35
good, what about the rest of the list? How large is your database? does it fit in memory and how large is your mongodb cluster?
– petrch
Nov 16 '18 at 13:36
2.5 GB of total used space is 1.75GB
– Aravindh
Nov 16 '18 at 13:39
can you provide the find command or the pipeline query and output of the explain for that same command?
– petrch
Nov 16 '18 at 14:40
add a comment |
run db.collection.find().explain() and make sure that your index is in fact used. Make sure that you do not have COLLSCANs there https://docs.mongodb.com/manual/reference/explain-results/.
if your documents have some/many large attributes and you need only some attributes try to request only them (e.g. only _id or _id and name). Less data transferred gives higher speed.
if your db does not fit in memory, make it fit in memory. Once the database does not fit the performance will be much worse.
if you are not running on a sharded cluster, create one based on a reasonable sharding key. Age may not be a good one because than all age=25 documents will end up on one node. Even if you have one computer with multiple CPUs it still may work better for you (if you have enough memory for that). It may even work the other way around. If you have a sharded cluster on one computer and your replicas do not fit in the memory, it may be better to use just one node.
answered Nov 16 '18 at 13:00
petrchpetrch
32627
I checked with db.collection.find().explain(), and I am sure that my indexing is working I used only the required fields i.e., upto 7 fields and I am using shared cluster
– Aravindh
Nov 16 '18 at 13:35
good, what about the rest of the list? How large is your database? does it fit in memory and how large is your mongodb cluster?
– petrch
Nov 16 '18 at 13:36
2.5 GB of total used space is 1.75GB
– Aravindh
Nov 16 '18 at 13:39
can you provide the find command or the pipeline query and output of the explain for that same command?
– petrch
Nov 16 '18 at 14:40
add a comment |
run db.collection.find().explain() and make sure that your index is in fact used. Make sure that you do not have COLLSCANs there https://docs.mongodb.com/manual/reference/explain-results/.
if your documents have some/many large attributes and you need only some attributes try to request only them (e.g. only _id or _id and name). Less data transferred gives higher speed.
if your db does not fit in memory, make it fit in memory. Once the database does not fit the performance will be much worse.
if you are not running on a sharded cluster, create one based on a reasonable sharding key. Age may not be a good one because than all age=25 documents will end up on one node. Even if you have one computer with multiple CPUs it still may work better for you (if you have enough memory for that). It may even work the other way around. If you have a sharded cluster on one computer and your replicas do not fit in the memory, it may be better to use just one node.
answered Nov 16 '18 at 13:00
petrchpetrch
32627
run db.collection.find().explain() and make sure that your index is in fact used. Make sure that you do not have COLLSCANs there https://docs.mongodb.com/manual/reference/explain-results/.
if your documents have some/many large attributes and you need only some attributes try to request only them (e.g. only _id or _id and name). Less data transferred gives higher speed.
if your db does not fit in memory, make it fit in memory. Once the database does not fit the performance will be much worse.
if you are not running on a sharded cluster, create one based on a reasonable sharding key. Age may not be a good one because than all age=25 documents will end up on one node. Even if you have one computer with multiple CPUs it still may work better for you (if you have enough memory for that). It may even work the other way around. If you have a sharded cluster on one computer and your replicas do not fit in the memory, it may be better to use just one node.
answered Nov 16 '18 at 13:00
petrchpetrch
32627
answered Nov 16 '18 at 13:00
petrchpetrch
32627
answered Nov 16 '18 at 13:00
petrchpetrch
32627
answered Nov 16 '18 at 13:00
petrchpetrch
32627
32627
I checked with db.collection.find().explain(), and I am sure that my indexing is working I used only the required fields i.e., upto 7 fields and I am using shared cluster
– Aravindh
Nov 16 '18 at 13:35
good, what about the rest of the list? How large is your database? does it fit in memory and how large is your mongodb cluster?
– petrch
Nov 16 '18 at 13:36
2.5 GB of total used space is 1.75GB
– Aravindh
Nov 16 '18 at 13:39
can you provide the find command or the pipeline query and output of the explain for that same command?
– petrch
Nov 16 '18 at 14:40
add a comment |
I checked with db.collection.find().explain(), and I am sure that my indexing is working I used only the required fields i.e., upto 7 fields and I am using shared cluster
– Aravindh
Nov 16 '18 at 13:35
good, what about the rest of the list? How large is your database? does it fit in memory and how large is your mongodb cluster?
– petrch
Nov 16 '18 at 13:36
2.5 GB of total used space is 1.75GB
– Aravindh
Nov 16 '18 at 13:39
can you provide the find command or the pipeline query and output of the explain for that same command?
– petrch
Nov 16 '18 at 14:40
I checked with db.collection.find().explain(), and I am sure that my indexing is working I used only the required fields i.e., upto 7 fields and I am using shared cluster
– Aravindh
Nov 16 '18 at 13:35
I checked with db.collection.find().explain(), and I am sure that my indexing is working I used only the required fields i.e., upto 7 fields and I am using shared cluster
– Aravindh
Nov 16 '18 at 13:35
good, what about the rest of the list? How large is your database? does it fit in memory and how large is your mongodb cluster?
– petrch
Nov 16 '18 at 13:36
good, what about the rest of the list? How large is your database? does it fit in memory and how large is your mongodb cluster?
– petrch
Nov 16 '18 at 13:36
2.5 GB of total used space is 1.75GB
– Aravindh
Nov 16 '18 at 13:39
2.5 GB of total used space is 1.75GB
– Aravindh
Nov 16 '18 at 13:39
can you provide the find command or the pipeline query and output of the explain for that same command?
– petrch
Nov 16 '18 at 14:40
can you provide the find command or the pipeline query and output of the explain for that same command?
– petrch
Nov 16 '18 at 14:40
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53338251%2fhow-to-reduce-the-querying-time-for-larger-size-data-in-mongodb%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
