“Request rate is large” while running mongorestore to CosmosDB
I'm attempting to dump and restore from my local Mongo DB to Azure Cosmos DB and I get the error "Request rate is large" My database is 9.3MB with 116 collections. I'm guessing by restoring collection by collection it will work. Is this the only way? Or to move to next pricing tier?
azure-cosmosdb edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
add a comment |
I'm attempting to dump and restore from my local Mongo DB to Azure Cosmos DB and I get the error "Request rate is large" My database is 9.3MB with 116 collections. I'm guessing by restoring collection by collection it will work. Is this the only way? Or to move to next pricing tier?
azure-cosmosdb edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
add a comment |
I'm attempting to dump and restore from my local Mongo DB to Azure Cosmos DB and I get the error "Request rate is large" My database is 9.3MB with 116 collections. I'm guessing by restoring collection by collection it will work. Is this the only way? Or to move to next pricing tier?
azure-cosmosdb edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
I'm attempting to dump and restore from my local Mongo DB to Azure Cosmos DB and I get the error "Request rate is large" My database is 9.3MB with 116 collections. I'm guessing by restoring collection by collection it will work. Is this the only way? Or to move to next pricing tier?
azure-cosmosdb azure-cosmosdb edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
edited Nov 14 '18 at 21:21
Neil Lunn
98.5k23174184
98.5k23174184
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
asked Nov 14 '18 at 16:50
PrimicoPrimico
3871618
3871618
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In Cosmos DB there are no pricing tiers, but rather provisioned throughput on the collection and less commonly, database level.
The reason why you are getting a 429 Request rate is large error is because you are hitting CosmosDB with more RU/s than the ones provisioned. This has nothing to do with the volume of your database but rather the request rate you are hitting cosmos with.
You can prevent this from happening by increasing the provisioned throughput from the Scale settings in the Azure portal, or increasing the retries in case of a throttled request on the SDK level.
The increase can be done momentarily in order to import the data and then scale it back down.
However, having 116 collections for 9.3MB of data is not a good idea in CosmosDB as you will be charged for a minimum of 400 RU/s per collection. I would recommend you read more about CosmosDB pricing and CosmosDB errors.
Error codes: https://docs.microsoft.com/en-us/rest/api/cosmos-db/http-status-codes-for-cosmosdb
Pricing: https://azure.microsoft.com/en-gb/pricing/details/cosmos-db/
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
Thanks for the very thorough explanation. I am testing an open source ecommerce solution grandnode.com that uses MongoDB and has 116 collections. It sounds like the pricing structure makes hosting this website in Azure even more expensive than a similar solution that uses SQL? It was under the impression that these NoSQL solutions were cheaper.
– Primico
Nov 14 '18 at 17:58
Like Nick said, for the scale of your solution is a very expensive platform, you could instead try: Use a managed mongoDB from other suppliers like mongoDB atlas, or host a mongoDB database in an Azure VM, will be much cheaper but will add extra work to manage, backup replication and so on.
– Diego Mendes
Nov 14 '18 at 21:58
@Primico - For something with so many collections, consider database-level Request Units, where all collections in the database share from an RU pool, so that you don't have to pay the minimum cost for each collection.
– David Makogon
Nov 15 '18 at 0:32
@DiegoMendes - this is what database-level RU is for, and the minimum cost is a fraction of having 116 collection-level RU minimums.
– David Makogon
Nov 15 '18 at 0:34
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305135%2frequest-rate-is-large-while-running-mongorestore-to-cosmosdb%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In Cosmos DB there are no pricing tiers, but rather provisioned throughput on the collection and less commonly, database level.
The reason why you are getting a 429 Request rate is large error is because you are hitting CosmosDB with more RU/s than the ones provisioned. This has nothing to do with the volume of your database but rather the request rate you are hitting cosmos with.
You can prevent this from happening by increasing the provisioned throughput from the Scale settings in the Azure portal, or increasing the retries in case of a throttled request on the SDK level.
The increase can be done momentarily in order to import the data and then scale it back down.
However, having 116 collections for 9.3MB of data is not a good idea in CosmosDB as you will be charged for a minimum of 400 RU/s per collection. I would recommend you read more about CosmosDB pricing and CosmosDB errors.
Error codes: https://docs.microsoft.com/en-us/rest/api/cosmos-db/http-status-codes-for-cosmosdb
Pricing: https://azure.microsoft.com/en-gb/pricing/details/cosmos-db/
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
Thanks for the very thorough explanation. I am testing an open source ecommerce solution grandnode.com that uses MongoDB and has 116 collections. It sounds like the pricing structure makes hosting this website in Azure even more expensive than a similar solution that uses SQL? It was under the impression that these NoSQL solutions were cheaper.
– Primico
Nov 14 '18 at 17:58
Like Nick said, for the scale of your solution is a very expensive platform, you could instead try: Use a managed mongoDB from other suppliers like mongoDB atlas, or host a mongoDB database in an Azure VM, will be much cheaper but will add extra work to manage, backup replication and so on.
– Diego Mendes
Nov 14 '18 at 21:58
@Primico - For something with so many collections, consider database-level Request Units, where all collections in the database share from an RU pool, so that you don't have to pay the minimum cost for each collection.
– David Makogon
Nov 15 '18 at 0:32
@DiegoMendes - this is what database-level RU is for, and the minimum cost is a fraction of having 116 collection-level RU minimums.
– David Makogon
Nov 15 '18 at 0:34
add a comment |
In Cosmos DB there are no pricing tiers, but rather provisioned throughput on the collection and less commonly, database level.
The reason why you are getting a 429 Request rate is large error is because you are hitting CosmosDB with more RU/s than the ones provisioned. This has nothing to do with the volume of your database but rather the request rate you are hitting cosmos with.
You can prevent this from happening by increasing the provisioned throughput from the Scale settings in the Azure portal, or increasing the retries in case of a throttled request on the SDK level.
The increase can be done momentarily in order to import the data and then scale it back down.
However, having 116 collections for 9.3MB of data is not a good idea in CosmosDB as you will be charged for a minimum of 400 RU/s per collection. I would recommend you read more about CosmosDB pricing and CosmosDB errors.
Error codes: https://docs.microsoft.com/en-us/rest/api/cosmos-db/http-status-codes-for-cosmosdb
Pricing: https://azure.microsoft.com/en-gb/pricing/details/cosmos-db/
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
Thanks for the very thorough explanation. I am testing an open source ecommerce solution grandnode.com that uses MongoDB and has 116 collections. It sounds like the pricing structure makes hosting this website in Azure even more expensive than a similar solution that uses SQL? It was under the impression that these NoSQL solutions were cheaper.
– Primico
Nov 14 '18 at 17:58
Like Nick said, for the scale of your solution is a very expensive platform, you could instead try: Use a managed mongoDB from other suppliers like mongoDB atlas, or host a mongoDB database in an Azure VM, will be much cheaper but will add extra work to manage, backup replication and so on.
– Diego Mendes
Nov 14 '18 at 21:58
@Primico - For something with so many collections, consider database-level Request Units, where all collections in the database share from an RU pool, so that you don't have to pay the minimum cost for each collection.
– David Makogon
Nov 15 '18 at 0:32
@DiegoMendes - this is what database-level RU is for, and the minimum cost is a fraction of having 116 collection-level RU minimums.
– David Makogon
Nov 15 '18 at 0:34
add a comment |
In Cosmos DB there are no pricing tiers, but rather provisioned throughput on the collection and less commonly, database level.
The reason why you are getting a 429 Request rate is large error is because you are hitting CosmosDB with more RU/s than the ones provisioned. This has nothing to do with the volume of your database but rather the request rate you are hitting cosmos with.
You can prevent this from happening by increasing the provisioned throughput from the Scale settings in the Azure portal, or increasing the retries in case of a throttled request on the SDK level.
The increase can be done momentarily in order to import the data and then scale it back down.
However, having 116 collections for 9.3MB of data is not a good idea in CosmosDB as you will be charged for a minimum of 400 RU/s per collection. I would recommend you read more about CosmosDB pricing and CosmosDB errors.
Error codes: https://docs.microsoft.com/en-us/rest/api/cosmos-db/http-status-codes-for-cosmosdb
Pricing: https://azure.microsoft.com/en-gb/pricing/details/cosmos-db/
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
In Cosmos DB there are no pricing tiers, but rather provisioned throughput on the collection and less commonly, database level.
The reason why you are getting a 429 Request rate is large error is because you are hitting CosmosDB with more RU/s than the ones provisioned. This has nothing to do with the volume of your database but rather the request rate you are hitting cosmos with.
You can prevent this from happening by increasing the provisioned throughput from the Scale settings in the Azure portal, or increasing the retries in case of a throttled request on the SDK level.
The increase can be done momentarily in order to import the data and then scale it back down.
However, having 116 collections for 9.3MB of data is not a good idea in CosmosDB as you will be charged for a minimum of 400 RU/s per collection. I would recommend you read more about CosmosDB pricing and CosmosDB errors.
Error codes: https://docs.microsoft.com/en-us/rest/api/cosmos-db/http-status-codes-for-cosmosdb
Pricing: https://azure.microsoft.com/en-gb/pricing/details/cosmos-db/
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
answered Nov 14 '18 at 17:01
Nick ChapsasNick Chapsas
2,9211514
2,9211514
Thanks for the very thorough explanation. I am testing an open source ecommerce solution grandnode.com that uses MongoDB and has 116 collections. It sounds like the pricing structure makes hosting this website in Azure even more expensive than a similar solution that uses SQL? It was under the impression that these NoSQL solutions were cheaper.
– Primico
Nov 14 '18 at 17:58
Like Nick said, for the scale of your solution is a very expensive platform, you could instead try: Use a managed mongoDB from other suppliers like mongoDB atlas, or host a mongoDB database in an Azure VM, will be much cheaper but will add extra work to manage, backup replication and so on.
– Diego Mendes
Nov 14 '18 at 21:58
@Primico - For something with so many collections, consider database-level Request Units, where all collections in the database share from an RU pool, so that you don't have to pay the minimum cost for each collection.
– David Makogon
Nov 15 '18 at 0:32
@DiegoMendes - this is what database-level RU is for, and the minimum cost is a fraction of having 116 collection-level RU minimums.
– David Makogon
Nov 15 '18 at 0:34
add a comment |
Thanks for the very thorough explanation. I am testing an open source ecommerce solution grandnode.com that uses MongoDB and has 116 collections. It sounds like the pricing structure makes hosting this website in Azure even more expensive than a similar solution that uses SQL? It was under the impression that these NoSQL solutions were cheaper.
– Primico
Nov 14 '18 at 17:58
Like Nick said, for the scale of your solution is a very expensive platform, you could instead try: Use a managed mongoDB from other suppliers like mongoDB atlas, or host a mongoDB database in an Azure VM, will be much cheaper but will add extra work to manage, backup replication and so on.
– Diego Mendes
Nov 14 '18 at 21:58
@Primico - For something with so many collections, consider database-level Request Units, where all collections in the database share from an RU pool, so that you don't have to pay the minimum cost for each collection.
– David Makogon
Nov 15 '18 at 0:32
@DiegoMendes - this is what database-level RU is for, and the minimum cost is a fraction of having 116 collection-level RU minimums.
– David Makogon
Nov 15 '18 at 0:34
Thanks for the very thorough explanation. I am testing an open source ecommerce solution grandnode.com that uses MongoDB and has 116 collections. It sounds like the pricing structure makes hosting this website in Azure even more expensive than a similar solution that uses SQL? It was under the impression that these NoSQL solutions were cheaper.
– Primico
Nov 14 '18 at 17:58
Thanks for the very thorough explanation. I am testing an open source ecommerce solution grandnode.com that uses MongoDB and has 116 collections. It sounds like the pricing structure makes hosting this website in Azure even more expensive than a similar solution that uses SQL? It was under the impression that these NoSQL solutions were cheaper.
– Primico
Nov 14 '18 at 17:58
Like Nick said, for the scale of your solution is a very expensive platform, you could instead try: Use a managed mongoDB from other suppliers like mongoDB atlas, or host a mongoDB database in an Azure VM, will be much cheaper but will add extra work to manage, backup replication and so on.
– Diego Mendes
Nov 14 '18 at 21:58
Like Nick said, for the scale of your solution is a very expensive platform, you could instead try: Use a managed mongoDB from other suppliers like mongoDB atlas, or host a mongoDB database in an Azure VM, will be much cheaper but will add extra work to manage, backup replication and so on.
– Diego Mendes
Nov 14 '18 at 21:58
@Primico - For something with so many collections, consider database-level Request Units, where all collections in the database share from an RU pool, so that you don't have to pay the minimum cost for each collection.
– David Makogon
Nov 15 '18 at 0:32
@Primico - For something with so many collections, consider database-level Request Units, where all collections in the database share from an RU pool, so that you don't have to pay the minimum cost for each collection.
– David Makogon
Nov 15 '18 at 0:32
@DiegoMendes - this is what database-level RU is for, and the minimum cost is a fraction of having 116 collection-level RU minimums.
– David Makogon
Nov 15 '18 at 0:34
@DiegoMendes - this is what database-level RU is for, and the minimum cost is a fraction of having 116 collection-level RU minimums.
– David Makogon
Nov 15 '18 at 0:34
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305135%2frequest-rate-is-large-while-running-mongorestore-to-cosmosdb%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
