Python Lambda function to capture AWS cloud watch logs of AWS MQ and send to kinesis
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
I got an Python script from put_records() only accepts keyword arguments in Kinesis boto3 Python API which will load json files to kinesis stream.
My architecture is something like this
In AWS console, I have created a Lambda function with the above code added to it.Lambda Function
How will I integrate or tell my lambda function that it wakes up every one minute. Should I need to add the capture messages via Cloud-watch events. If so how..?
I did get this solution form below link.
Python Script:-
import time
import boto3
import stomp
kinesis_client = boto3.client('kinesis')
class Listener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
kinesis_client.put_record(
StreamName='inter-lambda',
Data=u'{}rn'.format(message).encode('utf-8'),
PartitionKey='0'
)
def handler(event, context):
conn = stomp.Connection(host_and_ports=[('localhost', 61616)])
conn.set_listener('', Listener(conn))
conn.start()
conn.connect(login='user', passcode='pass')
conn.subscribe(destination='A.B.C.D', ack='auto')
print('Waiting for messages...')
time.sleep(10)
conn.close()
return ''
https://github.com/aws-samples/amazonmq-invoke-aws-lambda
amazon-web-services aws-lambda amazon-kinesis amazon-cloudwatchlogs amazon-mq
add a comment |
I got an Python script from put_records() only accepts keyword arguments in Kinesis boto3 Python API which will load json files to kinesis stream.
My architecture is something like this
In AWS console, I have created a Lambda function with the above code added to it.Lambda Function
How will I integrate or tell my lambda function that it wakes up every one minute. Should I need to add the capture messages via Cloud-watch events. If so how..?
I did get this solution form below link.
Python Script:-
import time
import boto3
import stomp
kinesis_client = boto3.client('kinesis')
class Listener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
kinesis_client.put_record(
StreamName='inter-lambda',
Data=u'{}rn'.format(message).encode('utf-8'),
PartitionKey='0'
)
def handler(event, context):
conn = stomp.Connection(host_and_ports=[('localhost', 61616)])
conn.set_listener('', Listener(conn))
conn.start()
conn.connect(login='user', passcode='pass')
conn.subscribe(destination='A.B.C.D', ack='auto')
print('Waiting for messages...')
time.sleep(10)
conn.close()
return ''
https://github.com/aws-samples/amazonmq-invoke-aws-lambda
amazon-web-services aws-lambda amazon-kinesis amazon-cloudwatchlogs amazon-mq
add a comment |
I got an Python script from put_records() only accepts keyword arguments in Kinesis boto3 Python API which will load json files to kinesis stream.
My architecture is something like this
In AWS console, I have created a Lambda function with the above code added to it.Lambda Function
How will I integrate or tell my lambda function that it wakes up every one minute. Should I need to add the capture messages via Cloud-watch events. If so how..?
I did get this solution form below link.
Python Script:-
import time
import boto3
import stomp
kinesis_client = boto3.client('kinesis')
class Listener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
kinesis_client.put_record(
StreamName='inter-lambda',
Data=u'{}rn'.format(message).encode('utf-8'),
PartitionKey='0'
)
def handler(event, context):
conn = stomp.Connection(host_and_ports=[('localhost', 61616)])
conn.set_listener('', Listener(conn))
conn.start()
conn.connect(login='user', passcode='pass')
conn.subscribe(destination='A.B.C.D', ack='auto')
print('Waiting for messages...')
time.sleep(10)
conn.close()
return ''
https://github.com/aws-samples/amazonmq-invoke-aws-lambda
amazon-web-services aws-lambda amazon-kinesis amazon-cloudwatchlogs amazon-mq
I got an Python script from put_records() only accepts keyword arguments in Kinesis boto3 Python API which will load json files to kinesis stream.
My architecture is something like this
In AWS console, I have created a Lambda function with the above code added to it.Lambda Function
How will I integrate or tell my lambda function that it wakes up every one minute. Should I need to add the capture messages via Cloud-watch events. If so how..?
I did get this solution form below link.
Python Script:-
import time
import boto3
import stomp
kinesis_client = boto3.client('kinesis')
class Listener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
kinesis_client.put_record(
StreamName='inter-lambda',
Data=u'{}rn'.format(message).encode('utf-8'),
PartitionKey='0'
)
def handler(event, context):
conn = stomp.Connection(host_and_ports=[('localhost', 61616)])
conn.set_listener('', Listener(conn))
conn.start()
conn.connect(login='user', passcode='pass')
conn.subscribe(destination='A.B.C.D', ack='auto')
print('Waiting for messages...')
time.sleep(10)
conn.close()
return ''
https://github.com/aws-samples/amazonmq-invoke-aws-lambda
amazon-web-services aws-lambda amazon-kinesis amazon-cloudwatchlogs amazon-mq
amazon-web-services aws-lambda amazon-kinesis amazon-cloudwatchlogs amazon-mq
edited Nov 17 '18 at 19:24
asked Nov 16 '18 at 17:26
user10664125
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You can schedule Lambda functions to run using CloudWatch Events.
Another alternative may be to subscribe to the log events and deliver them to Lambda.
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
Thank you..! But how will my Lambda function captures incoming MQ messages and send back to Kinesis
– user10664125
Nov 18 '18 at 1:34
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53342674%2fpython-lambda-function-to-capture-aws-cloud-watch-logs-of-aws-mq-and-send-to-kin%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can schedule Lambda functions to run using CloudWatch Events.
Another alternative may be to subscribe to the log events and deliver them to Lambda.
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
Thank you..! But how will my Lambda function captures incoming MQ messages and send back to Kinesis
– user10664125
Nov 18 '18 at 1:34
add a comment |
You can schedule Lambda functions to run using CloudWatch Events.
Another alternative may be to subscribe to the log events and deliver them to Lambda.
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
Thank you..! But how will my Lambda function captures incoming MQ messages and send back to Kinesis
– user10664125
Nov 18 '18 at 1:34
add a comment |
You can schedule Lambda functions to run using CloudWatch Events.
Another alternative may be to subscribe to the log events and deliver them to Lambda.
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
You can schedule Lambda functions to run using CloudWatch Events.
Another alternative may be to subscribe to the log events and deliver them to Lambda.
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
answered Nov 17 '18 at 21:45
jarmodjarmod
20.8k74149
20.8k74149
Thank you..! But how will my Lambda function captures incoming MQ messages and send back to Kinesis
– user10664125
Nov 18 '18 at 1:34
add a comment |
Thank you..! But how will my Lambda function captures incoming MQ messages and send back to Kinesis
– user10664125
Nov 18 '18 at 1:34
Thank you..! But how will my Lambda function captures incoming MQ messages and send back to Kinesis
– user10664125
Nov 18 '18 at 1:34
Thank you..! But how will my Lambda function captures incoming MQ messages and send back to Kinesis
– user10664125
Nov 18 '18 at 1:34
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53342674%2fpython-lambda-function-to-capture-aws-cloud-watch-logs-of-aws-mq-and-send-to-kin%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
