Messages disappearing from SQS queue to Lambda
I am sending messages from an SQS queue to a Node.js Lambda function. I send about 20 messages at the same time to the queue after which Lambda starts working.
Some of these messages never seem to appear in the queue, or at least don't make it to Lambda. I don't see any activity in the Cloudwatch logs for those missing messages.
It's a long polling queue with these settings:
- Default Visibility Timeout: 2 minutes
- Delivery delay: 0 seconds
- Receive Message Wait Time: 5 seconds
- Queue type: Standard
The Lambda script finished in a few seconds and has concurrency set to 'Use unreserved account concurrency'.
For debugging purposes, I'm now also sending all the messages to a second queue that doesn't get processed. There I can see all the messages, also the ones that are missing from the other one.
The messages just contain about 3 numeric variables in a json, so the size shouldn't be an issue.
I have set up a redrive policy, and the missing messages don't show up in the dead letter queue either.
Kinda pulling my hair out here. Any ideas why these messages seem to disappear?
amazon-web-services aws-lambda amazon-sqs
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
asked Nov 16 '18 at 11:55
RickRick
3081414
add a comment |
I am sending messages from an SQS queue to a Node.js Lambda function. I send about 20 messages at the same time to the queue after which Lambda starts working.
Some of these messages never seem to appear in the queue, or at least don't make it to Lambda. I don't see any activity in the Cloudwatch logs for those missing messages.
It's a long polling queue with these settings:
- Default Visibility Timeout: 2 minutes
- Delivery delay: 0 seconds
- Receive Message Wait Time: 5 seconds
- Queue type: Standard
The Lambda script finished in a few seconds and has concurrency set to 'Use unreserved account concurrency'.
For debugging purposes, I'm now also sending all the messages to a second queue that doesn't get processed. There I can see all the messages, also the ones that are missing from the other one.
The messages just contain about 3 numeric variables in a json, so the size shouldn't be an issue.
I have set up a redrive policy, and the missing messages don't show up in the dead letter queue either.
Kinda pulling my hair out here. Any ideas why these messages seem to disappear?
amazon-web-services aws-lambda amazon-sqs
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
asked Nov 16 '18 at 11:55
RickRick
3081414
1
Could you include the result of logging the event that your lambda is invoked with?
– thomasmichaelwallace
Nov 16 '18 at 13:38
3
My guess is multiple messages are being sent to a single Lambda invocation (which is the default setting, and you didn't include the Batch Size setting in your question) and your Lambda function is only handling the first message instead of properly looping through all the messages passed to it.
– Mark B
Nov 16 '18 at 14:30
Setting the Batch Size to 1 fixes things.
– Rick
Nov 16 '18 at 20:21
Or you could simply add a loop into your Lambda function, process all N messages sequentially, and hopefully save some time and money.
– jarmod
Nov 17 '18 at 22:01
Good one, I'll do that.
– Rick
Nov 18 '18 at 19:19
add a comment |
I am sending messages from an SQS queue to a Node.js Lambda function. I send about 20 messages at the same time to the queue after which Lambda starts working.
Some of these messages never seem to appear in the queue, or at least don't make it to Lambda. I don't see any activity in the Cloudwatch logs for those missing messages.
It's a long polling queue with these settings:
- Default Visibility Timeout: 2 minutes
- Delivery delay: 0 seconds
- Receive Message Wait Time: 5 seconds
- Queue type: Standard
The Lambda script finished in a few seconds and has concurrency set to 'Use unreserved account concurrency'.
For debugging purposes, I'm now also sending all the messages to a second queue that doesn't get processed. There I can see all the messages, also the ones that are missing from the other one.
The messages just contain about 3 numeric variables in a json, so the size shouldn't be an issue.
I have set up a redrive policy, and the missing messages don't show up in the dead letter queue either.
Kinda pulling my hair out here. Any ideas why these messages seem to disappear?
amazon-web-services aws-lambda amazon-sqs
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
asked Nov 16 '18 at 11:55
RickRick
3081414
I am sending messages from an SQS queue to a Node.js Lambda function. I send about 20 messages at the same time to the queue after which Lambda starts working.
Some of these messages never seem to appear in the queue, or at least don't make it to Lambda. I don't see any activity in the Cloudwatch logs for those missing messages.
It's a long polling queue with these settings:
- Default Visibility Timeout: 2 minutes
- Delivery delay: 0 seconds
- Receive Message Wait Time: 5 seconds
- Queue type: Standard
The Lambda script finished in a few seconds and has concurrency set to 'Use unreserved account concurrency'.
For debugging purposes, I'm now also sending all the messages to a second queue that doesn't get processed. There I can see all the messages, also the ones that are missing from the other one.
The messages just contain about 3 numeric variables in a json, so the size shouldn't be an issue.
I have set up a redrive policy, and the missing messages don't show up in the dead letter queue either.
Kinda pulling my hair out here. Any ideas why these messages seem to disappear?
amazon-web-services aws-lambda amazon-sqs
amazon-web-services aws-lambda amazon-sqs
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
asked Nov 16 '18 at 11:55
RickRick
3081414
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
asked Nov 16 '18 at 11:55
RickRick
3081414
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
edited Nov 16 '18 at 16:32
John Rotenstein
77.7k787139
77.7k787139
asked Nov 16 '18 at 11:55
RickRick
3081414
asked Nov 16 '18 at 11:55
RickRick
3081414
asked Nov 16 '18 at 11:55
RickRick
3081414
3081414
1
Could you include the result of logging the event that your lambda is invoked with?
– thomasmichaelwallace
Nov 16 '18 at 13:38
3
My guess is multiple messages are being sent to a single Lambda invocation (which is the default setting, and you didn't include the Batch Size setting in your question) and your Lambda function is only handling the first message instead of properly looping through all the messages passed to it.
– Mark B
Nov 16 '18 at 14:30
Setting the Batch Size to 1 fixes things.
– Rick
Nov 16 '18 at 20:21
Or you could simply add a loop into your Lambda function, process all N messages sequentially, and hopefully save some time and money.
– jarmod
Nov 17 '18 at 22:01
Good one, I'll do that.
– Rick
Nov 18 '18 at 19:19
add a comment |
1
Could you include the result of logging the event that your lambda is invoked with?
– thomasmichaelwallace
Nov 16 '18 at 13:38
3
My guess is multiple messages are being sent to a single Lambda invocation (which is the default setting, and you didn't include the Batch Size setting in your question) and your Lambda function is only handling the first message instead of properly looping through all the messages passed to it.
– Mark B
Nov 16 '18 at 14:30
Setting the Batch Size to 1 fixes things.
– Rick
Nov 16 '18 at 20:21
Or you could simply add a loop into your Lambda function, process all N messages sequentially, and hopefully save some time and money.
– jarmod
Nov 17 '18 at 22:01
Good one, I'll do that.
– Rick
Nov 18 '18 at 19:19
1
1
Could you include the result of logging the event that your lambda is invoked with?
– thomasmichaelwallace
Nov 16 '18 at 13:38
Could you include the result of logging the event that your lambda is invoked with?
– thomasmichaelwallace
Nov 16 '18 at 13:38
3
3
My guess is multiple messages are being sent to a single Lambda invocation (which is the default setting, and you didn't include the Batch Size setting in your question) and your Lambda function is only handling the first message instead of properly looping through all the messages passed to it.
– Mark B
Nov 16 '18 at 14:30
My guess is multiple messages are being sent to a single Lambda invocation (which is the default setting, and you didn't include the Batch Size setting in your question) and your Lambda function is only handling the first message instead of properly looping through all the messages passed to it.
– Mark B
Nov 16 '18 at 14:30
Setting the Batch Size to 1 fixes things.
– Rick
Nov 16 '18 at 20:21
Setting the Batch Size to 1 fixes things.
– Rick
Nov 16 '18 at 20:21
Or you could simply add a loop into your Lambda function, process all N messages sequentially, and hopefully save some time and money.
– jarmod
Nov 17 '18 at 22:01
Or you could simply add a loop into your Lambda function, process all N messages sequentially, and hopefully save some time and money.
– jarmod
Nov 17 '18 at 22:01
Good one, I'll do that.
– Rick
Nov 18 '18 at 19:19
Good one, I'll do that.
– Rick
Nov 18 '18 at 19:19
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53337398%2fmessages-disappearing-from-sqs-queue-to-lambda%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53337398%2fmessages-disappearing-from-sqs-queue-to-lambda%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Could you include the result of logging the event that your lambda is invoked with?
– thomasmichaelwallace
Nov 16 '18 at 13:38
3
My guess is multiple messages are being sent to a single Lambda invocation (which is the default setting, and you didn't include the Batch Size setting in your question) and your Lambda function is only handling the first message instead of properly looping through all the messages passed to it.
– Mark B
Nov 16 '18 at 14:30
Setting the Batch Size to 1 fixes things.
– Rick
Nov 16 '18 at 20:21
Or you could simply add a loop into your Lambda function, process all N messages sequentially, and hopefully save some time and money.
– jarmod
Nov 17 '18 at 22:01
Good one, I'll do that.
– Rick
Nov 18 '18 at 19:19