“Fire and forget” python async/await
up vote
60
down vote
favorite
Sometimes there is some non-critical asynchronous operation that needs to happen but I don't want to wait for it to complete. In Tornado's coroutine implementation you can "fire & forget" an asynchronous function by simply ommitting the yield
key-word.
I've been trying to figure out how to "fire & forget" with the new async
/await
syntax released in Python 3.5. E.g., a simplified code snippet:
async def async_foo():
print("Do some stuff asynchronously here...")
def bar():
async_foo() # fire and forget "async_foo()"
bar()
What happens though is that bar()
never executes and instead we get a runtime warning:
RuntimeWarning: coroutine 'async_foo' was never awaited
async_foo() # fire and forget "async_foo()"
python python-3.5 python-asyncio
add a comment |
up vote
60
down vote
favorite
Sometimes there is some non-critical asynchronous operation that needs to happen but I don't want to wait for it to complete. In Tornado's coroutine implementation you can "fire & forget" an asynchronous function by simply ommitting the yield
key-word.
I've been trying to figure out how to "fire & forget" with the new async
/await
syntax released in Python 3.5. E.g., a simplified code snippet:
async def async_foo():
print("Do some stuff asynchronously here...")
def bar():
async_foo() # fire and forget "async_foo()"
bar()
What happens though is that bar()
never executes and instead we get a runtime warning:
RuntimeWarning: coroutine 'async_foo' was never awaited
async_foo() # fire and forget "async_foo()"
python python-3.5 python-asyncio
Related? stackoverflow.com/q/32808893/1639625 In fact, I think it's a duplicate, but I don't want to instant-dupe-hammer it. Can someone confirm?
– tobias_k
May 20 '16 at 10:00
3
@tobias_k, I don't think it's duplicate. Answer at the link is too broad to be answer for this question.
– Mikhail Gerasimov
May 20 '16 at 11:44
1
Does (1) your "main" process continue running forever ? Or (2) do you want to allow your process to die but allowing forgotten tasks continue their job ? Or (3) do you prefer your main process waiting for forgotten tasks just before ending ?
– Julien Palard
May 26 '16 at 8:23
add a comment |
up vote
60
down vote
favorite
up vote
60
down vote
favorite
Sometimes there is some non-critical asynchronous operation that needs to happen but I don't want to wait for it to complete. In Tornado's coroutine implementation you can "fire & forget" an asynchronous function by simply ommitting the yield
key-word.
I've been trying to figure out how to "fire & forget" with the new async
/await
syntax released in Python 3.5. E.g., a simplified code snippet:
async def async_foo():
print("Do some stuff asynchronously here...")
def bar():
async_foo() # fire and forget "async_foo()"
bar()
What happens though is that bar()
never executes and instead we get a runtime warning:
RuntimeWarning: coroutine 'async_foo' was never awaited
async_foo() # fire and forget "async_foo()"
python python-3.5 python-asyncio
Sometimes there is some non-critical asynchronous operation that needs to happen but I don't want to wait for it to complete. In Tornado's coroutine implementation you can "fire & forget" an asynchronous function by simply ommitting the yield
key-word.
I've been trying to figure out how to "fire & forget" with the new async
/await
syntax released in Python 3.5. E.g., a simplified code snippet:
async def async_foo():
print("Do some stuff asynchronously here...")
def bar():
async_foo() # fire and forget "async_foo()"
bar()
What happens though is that bar()
never executes and instead we get a runtime warning:
RuntimeWarning: coroutine 'async_foo' was never awaited
async_foo() # fire and forget "async_foo()"
python python-3.5 python-asyncio
python python-3.5 python-asyncio
edited May 20 '16 at 11:54
Mikhail Gerasimov
12.6k33561
12.6k33561
asked May 17 '16 at 14:13
Mike N
2,53921715
2,53921715
Related? stackoverflow.com/q/32808893/1639625 In fact, I think it's a duplicate, but I don't want to instant-dupe-hammer it. Can someone confirm?
– tobias_k
May 20 '16 at 10:00
3
@tobias_k, I don't think it's duplicate. Answer at the link is too broad to be answer for this question.
– Mikhail Gerasimov
May 20 '16 at 11:44
1
Does (1) your "main" process continue running forever ? Or (2) do you want to allow your process to die but allowing forgotten tasks continue their job ? Or (3) do you prefer your main process waiting for forgotten tasks just before ending ?
– Julien Palard
May 26 '16 at 8:23
add a comment |
Related? stackoverflow.com/q/32808893/1639625 In fact, I think it's a duplicate, but I don't want to instant-dupe-hammer it. Can someone confirm?
– tobias_k
May 20 '16 at 10:00
3
@tobias_k, I don't think it's duplicate. Answer at the link is too broad to be answer for this question.
– Mikhail Gerasimov
May 20 '16 at 11:44
1
Does (1) your "main" process continue running forever ? Or (2) do you want to allow your process to die but allowing forgotten tasks continue their job ? Or (3) do you prefer your main process waiting for forgotten tasks just before ending ?
– Julien Palard
May 26 '16 at 8:23
Related? stackoverflow.com/q/32808893/1639625 In fact, I think it's a duplicate, but I don't want to instant-dupe-hammer it. Can someone confirm?
– tobias_k
May 20 '16 at 10:00
Related? stackoverflow.com/q/32808893/1639625 In fact, I think it's a duplicate, but I don't want to instant-dupe-hammer it. Can someone confirm?
– tobias_k
May 20 '16 at 10:00
3
3
@tobias_k, I don't think it's duplicate. Answer at the link is too broad to be answer for this question.
– Mikhail Gerasimov
May 20 '16 at 11:44
@tobias_k, I don't think it's duplicate. Answer at the link is too broad to be answer for this question.
– Mikhail Gerasimov
May 20 '16 at 11:44
1
1
Does (1) your "main" process continue running forever ? Or (2) do you want to allow your process to die but allowing forgotten tasks continue their job ? Or (3) do you prefer your main process waiting for forgotten tasks just before ending ?
– Julien Palard
May 26 '16 at 8:23
Does (1) your "main" process continue running forever ? Or (2) do you want to allow your process to die but allowing forgotten tasks continue their job ? Or (3) do you prefer your main process waiting for forgotten tasks just before ending ?
– Julien Palard
May 26 '16 at 8:23
add a comment |
3 Answers
3
active
oldest
votes
up vote
96
down vote
accepted
Upd:
Replace asyncio.ensure_future
with asyncio.create_task
everywhere if you're using Python >= 3.7 It's newer, nicer way to spawn task.
asyncio.Task to “fire and forget”
According to python docs for asyncio.Task
it is possible to start some coroutine to execute "in background". The task created by asyncio.ensure_future
function won't block the execution (therefore the function will return immediately!). This looks like a way to “fire and forget” as you requested.
import asyncio
async def async_foo():
print("async_foo started")
await asyncio.sleep(1)
print("async_foo done")
async def main():
asyncio.ensure_future(async_foo()) # fire and forget async_foo()
# btw, you can also create tasks inside non-async funcs
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3
What if tasks are executing after event loop complete?
Note that asyncio expects task would be completed at the moment event loop completed. So if you'll change main()
to:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
You'll get this warning after the program finished:
Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]
To prevent that you can just await all pending tasks after event loop completed:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also finish all running tasks:
pending = asyncio.Task.all_tasks()
loop.run_until_complete(asyncio.gather(*pending))
Kill tasks instead of awaiting them
Sometimes you don't want to await tasks to be done (for example, some tasks may be created to run forever). In that case, you can just cancel() them instead of awaiting them:
import asyncio
from contextlib import suppress
async def echo_forever():
while True:
print("echo")
await asyncio.sleep(1)
async def main():
asyncio.ensure_future(echo_forever()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also cancel all running tasks:
pending = asyncio.Task.all_tasks()
for task in pending:
task.cancel()
# Now we should await task to execute it's cancellation.
# Cancelled task raises asyncio.CancelledError that we can suppress:
with suppress(asyncio.CancelledError):
loop.run_until_complete(task)
Output:
Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo
3
Wow, excellent and thorough answer. Thank you!
– Mike N
May 23 '16 at 13:44
I copied and past the first block and simply ran it on my end and for some reason I got: line 4 async def async_foo(): ^ As if there is some syntax error with the function definition on line 4: "async def async_foo():" Am I missing something?
– Gil Allen
Nov 6 '16 at 9:59
3
@GilAllen this syntax works only in Python 3.5+. Python 3.4 needs old syntax (see docs.python.org/3.4/library/asyncio-task.html ). Python 3.3 and below doesn't support asyncio at all.
– Mikhail Gerasimov
Nov 6 '16 at 14:14
@MikeN - thank you!
– Gil Allen
Nov 6 '16 at 14:22
How would you kill the tasks in a thread?…̣I have a thread that creates some tasks and I want to kill all the pending ones when the thread dies in itsstop()
method.
– Sardathrion
Nov 24 '17 at 15:21
|
show 4 more comments
up vote
4
down vote
This is not entirely asynchronous execution, but maybe run_in_executor() is suitable for you.
def fire_and_forget(task, *args, **kwargs):
loop = asyncio.get_event_loop()
if callable(task):
return loop.run_in_executor(None, task, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
def foo():
#asynchronous stuff here
fire_and_forget(foo)
Brilliant answer, Addresses the need of OP
– nehemiah
Nov 12 at 4:23
add a comment |
up vote
1
down vote
Thank you Sergey for the succint answer. Here is the decorated version of the same.
import asyncio
import time
def fire_and_forget(f):
from functools import wraps
@wraps(f)
def wrapped(*args, **kwargs):
loop = asyncio.get_event_loop()
if callable(f):
return loop.run_in_executor(None, f, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
return wrapped
@fire_and_forget
def foo():
print("foo() started")
time.sleep(1)
print("foo() completed")
print("Hello")
foo()
print("I didn't wait for foo()")
Produces
>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f37278647%2ffire-and-forget-python-async-await%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
96
down vote
accepted
Upd:
Replace asyncio.ensure_future
with asyncio.create_task
everywhere if you're using Python >= 3.7 It's newer, nicer way to spawn task.
asyncio.Task to “fire and forget”
According to python docs for asyncio.Task
it is possible to start some coroutine to execute "in background". The task created by asyncio.ensure_future
function won't block the execution (therefore the function will return immediately!). This looks like a way to “fire and forget” as you requested.
import asyncio
async def async_foo():
print("async_foo started")
await asyncio.sleep(1)
print("async_foo done")
async def main():
asyncio.ensure_future(async_foo()) # fire and forget async_foo()
# btw, you can also create tasks inside non-async funcs
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3
What if tasks are executing after event loop complete?
Note that asyncio expects task would be completed at the moment event loop completed. So if you'll change main()
to:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
You'll get this warning after the program finished:
Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]
To prevent that you can just await all pending tasks after event loop completed:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also finish all running tasks:
pending = asyncio.Task.all_tasks()
loop.run_until_complete(asyncio.gather(*pending))
Kill tasks instead of awaiting them
Sometimes you don't want to await tasks to be done (for example, some tasks may be created to run forever). In that case, you can just cancel() them instead of awaiting them:
import asyncio
from contextlib import suppress
async def echo_forever():
while True:
print("echo")
await asyncio.sleep(1)
async def main():
asyncio.ensure_future(echo_forever()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also cancel all running tasks:
pending = asyncio.Task.all_tasks()
for task in pending:
task.cancel()
# Now we should await task to execute it's cancellation.
# Cancelled task raises asyncio.CancelledError that we can suppress:
with suppress(asyncio.CancelledError):
loop.run_until_complete(task)
Output:
Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo
3
Wow, excellent and thorough answer. Thank you!
– Mike N
May 23 '16 at 13:44
I copied and past the first block and simply ran it on my end and for some reason I got: line 4 async def async_foo(): ^ As if there is some syntax error with the function definition on line 4: "async def async_foo():" Am I missing something?
– Gil Allen
Nov 6 '16 at 9:59
3
@GilAllen this syntax works only in Python 3.5+. Python 3.4 needs old syntax (see docs.python.org/3.4/library/asyncio-task.html ). Python 3.3 and below doesn't support asyncio at all.
– Mikhail Gerasimov
Nov 6 '16 at 14:14
@MikeN - thank you!
– Gil Allen
Nov 6 '16 at 14:22
How would you kill the tasks in a thread?…̣I have a thread that creates some tasks and I want to kill all the pending ones when the thread dies in itsstop()
method.
– Sardathrion
Nov 24 '17 at 15:21
|
show 4 more comments
up vote
96
down vote
accepted
Upd:
Replace asyncio.ensure_future
with asyncio.create_task
everywhere if you're using Python >= 3.7 It's newer, nicer way to spawn task.
asyncio.Task to “fire and forget”
According to python docs for asyncio.Task
it is possible to start some coroutine to execute "in background". The task created by asyncio.ensure_future
function won't block the execution (therefore the function will return immediately!). This looks like a way to “fire and forget” as you requested.
import asyncio
async def async_foo():
print("async_foo started")
await asyncio.sleep(1)
print("async_foo done")
async def main():
asyncio.ensure_future(async_foo()) # fire and forget async_foo()
# btw, you can also create tasks inside non-async funcs
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3
What if tasks are executing after event loop complete?
Note that asyncio expects task would be completed at the moment event loop completed. So if you'll change main()
to:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
You'll get this warning after the program finished:
Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]
To prevent that you can just await all pending tasks after event loop completed:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also finish all running tasks:
pending = asyncio.Task.all_tasks()
loop.run_until_complete(asyncio.gather(*pending))
Kill tasks instead of awaiting them
Sometimes you don't want to await tasks to be done (for example, some tasks may be created to run forever). In that case, you can just cancel() them instead of awaiting them:
import asyncio
from contextlib import suppress
async def echo_forever():
while True:
print("echo")
await asyncio.sleep(1)
async def main():
asyncio.ensure_future(echo_forever()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also cancel all running tasks:
pending = asyncio.Task.all_tasks()
for task in pending:
task.cancel()
# Now we should await task to execute it's cancellation.
# Cancelled task raises asyncio.CancelledError that we can suppress:
with suppress(asyncio.CancelledError):
loop.run_until_complete(task)
Output:
Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo
3
Wow, excellent and thorough answer. Thank you!
– Mike N
May 23 '16 at 13:44
I copied and past the first block and simply ran it on my end and for some reason I got: line 4 async def async_foo(): ^ As if there is some syntax error with the function definition on line 4: "async def async_foo():" Am I missing something?
– Gil Allen
Nov 6 '16 at 9:59
3
@GilAllen this syntax works only in Python 3.5+. Python 3.4 needs old syntax (see docs.python.org/3.4/library/asyncio-task.html ). Python 3.3 and below doesn't support asyncio at all.
– Mikhail Gerasimov
Nov 6 '16 at 14:14
@MikeN - thank you!
– Gil Allen
Nov 6 '16 at 14:22
How would you kill the tasks in a thread?…̣I have a thread that creates some tasks and I want to kill all the pending ones when the thread dies in itsstop()
method.
– Sardathrion
Nov 24 '17 at 15:21
|
show 4 more comments
up vote
96
down vote
accepted
up vote
96
down vote
accepted
Upd:
Replace asyncio.ensure_future
with asyncio.create_task
everywhere if you're using Python >= 3.7 It's newer, nicer way to spawn task.
asyncio.Task to “fire and forget”
According to python docs for asyncio.Task
it is possible to start some coroutine to execute "in background". The task created by asyncio.ensure_future
function won't block the execution (therefore the function will return immediately!). This looks like a way to “fire and forget” as you requested.
import asyncio
async def async_foo():
print("async_foo started")
await asyncio.sleep(1)
print("async_foo done")
async def main():
asyncio.ensure_future(async_foo()) # fire and forget async_foo()
# btw, you can also create tasks inside non-async funcs
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3
What if tasks are executing after event loop complete?
Note that asyncio expects task would be completed at the moment event loop completed. So if you'll change main()
to:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
You'll get this warning after the program finished:
Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]
To prevent that you can just await all pending tasks after event loop completed:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also finish all running tasks:
pending = asyncio.Task.all_tasks()
loop.run_until_complete(asyncio.gather(*pending))
Kill tasks instead of awaiting them
Sometimes you don't want to await tasks to be done (for example, some tasks may be created to run forever). In that case, you can just cancel() them instead of awaiting them:
import asyncio
from contextlib import suppress
async def echo_forever():
while True:
print("echo")
await asyncio.sleep(1)
async def main():
asyncio.ensure_future(echo_forever()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also cancel all running tasks:
pending = asyncio.Task.all_tasks()
for task in pending:
task.cancel()
# Now we should await task to execute it's cancellation.
# Cancelled task raises asyncio.CancelledError that we can suppress:
with suppress(asyncio.CancelledError):
loop.run_until_complete(task)
Output:
Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo
Upd:
Replace asyncio.ensure_future
with asyncio.create_task
everywhere if you're using Python >= 3.7 It's newer, nicer way to spawn task.
asyncio.Task to “fire and forget”
According to python docs for asyncio.Task
it is possible to start some coroutine to execute "in background". The task created by asyncio.ensure_future
function won't block the execution (therefore the function will return immediately!). This looks like a way to “fire and forget” as you requested.
import asyncio
async def async_foo():
print("async_foo started")
await asyncio.sleep(1)
print("async_foo done")
async def main():
asyncio.ensure_future(async_foo()) # fire and forget async_foo()
# btw, you can also create tasks inside non-async funcs
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3
What if tasks are executing after event loop complete?
Note that asyncio expects task would be completed at the moment event loop completed. So if you'll change main()
to:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
You'll get this warning after the program finished:
Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]
To prevent that you can just await all pending tasks after event loop completed:
async def main():
asyncio.ensure_future(async_foo()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(0.1)
print('Do some actions 2')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also finish all running tasks:
pending = asyncio.Task.all_tasks()
loop.run_until_complete(asyncio.gather(*pending))
Kill tasks instead of awaiting them
Sometimes you don't want to await tasks to be done (for example, some tasks may be created to run forever). In that case, you can just cancel() them instead of awaiting them:
import asyncio
from contextlib import suppress
async def echo_forever():
while True:
print("echo")
await asyncio.sleep(1)
async def main():
asyncio.ensure_future(echo_forever()) # fire and forget
print('Do some actions 1')
await asyncio.sleep(1)
print('Do some actions 2')
await asyncio.sleep(1)
print('Do some actions 3')
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
# Let's also cancel all running tasks:
pending = asyncio.Task.all_tasks()
for task in pending:
task.cancel()
# Now we should await task to execute it's cancellation.
# Cancelled task raises asyncio.CancelledError that we can suppress:
with suppress(asyncio.CancelledError):
loop.run_until_complete(task)
Output:
Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo
edited Dec 1 at 17:21
answered May 20 '16 at 11:30
Mikhail Gerasimov
12.6k33561
12.6k33561
3
Wow, excellent and thorough answer. Thank you!
– Mike N
May 23 '16 at 13:44
I copied and past the first block and simply ran it on my end and for some reason I got: line 4 async def async_foo(): ^ As if there is some syntax error with the function definition on line 4: "async def async_foo():" Am I missing something?
– Gil Allen
Nov 6 '16 at 9:59
3
@GilAllen this syntax works only in Python 3.5+. Python 3.4 needs old syntax (see docs.python.org/3.4/library/asyncio-task.html ). Python 3.3 and below doesn't support asyncio at all.
– Mikhail Gerasimov
Nov 6 '16 at 14:14
@MikeN - thank you!
– Gil Allen
Nov 6 '16 at 14:22
How would you kill the tasks in a thread?…̣I have a thread that creates some tasks and I want to kill all the pending ones when the thread dies in itsstop()
method.
– Sardathrion
Nov 24 '17 at 15:21
|
show 4 more comments
3
Wow, excellent and thorough answer. Thank you!
– Mike N
May 23 '16 at 13:44
I copied and past the first block and simply ran it on my end and for some reason I got: line 4 async def async_foo(): ^ As if there is some syntax error with the function definition on line 4: "async def async_foo():" Am I missing something?
– Gil Allen
Nov 6 '16 at 9:59
3
@GilAllen this syntax works only in Python 3.5+. Python 3.4 needs old syntax (see docs.python.org/3.4/library/asyncio-task.html ). Python 3.3 and below doesn't support asyncio at all.
– Mikhail Gerasimov
Nov 6 '16 at 14:14
@MikeN - thank you!
– Gil Allen
Nov 6 '16 at 14:22
How would you kill the tasks in a thread?…̣I have a thread that creates some tasks and I want to kill all the pending ones when the thread dies in itsstop()
method.
– Sardathrion
Nov 24 '17 at 15:21
3
3
Wow, excellent and thorough answer. Thank you!
– Mike N
May 23 '16 at 13:44
Wow, excellent and thorough answer. Thank you!
– Mike N
May 23 '16 at 13:44
I copied and past the first block and simply ran it on my end and for some reason I got: line 4 async def async_foo(): ^ As if there is some syntax error with the function definition on line 4: "async def async_foo():" Am I missing something?
– Gil Allen
Nov 6 '16 at 9:59
I copied and past the first block and simply ran it on my end and for some reason I got: line 4 async def async_foo(): ^ As if there is some syntax error with the function definition on line 4: "async def async_foo():" Am I missing something?
– Gil Allen
Nov 6 '16 at 9:59
3
3
@GilAllen this syntax works only in Python 3.5+. Python 3.4 needs old syntax (see docs.python.org/3.4/library/asyncio-task.html ). Python 3.3 and below doesn't support asyncio at all.
– Mikhail Gerasimov
Nov 6 '16 at 14:14
@GilAllen this syntax works only in Python 3.5+. Python 3.4 needs old syntax (see docs.python.org/3.4/library/asyncio-task.html ). Python 3.3 and below doesn't support asyncio at all.
– Mikhail Gerasimov
Nov 6 '16 at 14:14
@MikeN - thank you!
– Gil Allen
Nov 6 '16 at 14:22
@MikeN - thank you!
– Gil Allen
Nov 6 '16 at 14:22
How would you kill the tasks in a thread?…̣I have a thread that creates some tasks and I want to kill all the pending ones when the thread dies in its
stop()
method.– Sardathrion
Nov 24 '17 at 15:21
How would you kill the tasks in a thread?…̣I have a thread that creates some tasks and I want to kill all the pending ones when the thread dies in its
stop()
method.– Sardathrion
Nov 24 '17 at 15:21
|
show 4 more comments
up vote
4
down vote
This is not entirely asynchronous execution, but maybe run_in_executor() is suitable for you.
def fire_and_forget(task, *args, **kwargs):
loop = asyncio.get_event_loop()
if callable(task):
return loop.run_in_executor(None, task, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
def foo():
#asynchronous stuff here
fire_and_forget(foo)
Brilliant answer, Addresses the need of OP
– nehemiah
Nov 12 at 4:23
add a comment |
up vote
4
down vote
This is not entirely asynchronous execution, but maybe run_in_executor() is suitable for you.
def fire_and_forget(task, *args, **kwargs):
loop = asyncio.get_event_loop()
if callable(task):
return loop.run_in_executor(None, task, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
def foo():
#asynchronous stuff here
fire_and_forget(foo)
Brilliant answer, Addresses the need of OP
– nehemiah
Nov 12 at 4:23
add a comment |
up vote
4
down vote
up vote
4
down vote
This is not entirely asynchronous execution, but maybe run_in_executor() is suitable for you.
def fire_and_forget(task, *args, **kwargs):
loop = asyncio.get_event_loop()
if callable(task):
return loop.run_in_executor(None, task, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
def foo():
#asynchronous stuff here
fire_and_forget(foo)
This is not entirely asynchronous execution, but maybe run_in_executor() is suitable for you.
def fire_and_forget(task, *args, **kwargs):
loop = asyncio.get_event_loop()
if callable(task):
return loop.run_in_executor(None, task, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
def foo():
#asynchronous stuff here
fire_and_forget(foo)
answered May 20 '16 at 10:59
Sergey Gornostaev
4,68921224
4,68921224
Brilliant answer, Addresses the need of OP
– nehemiah
Nov 12 at 4:23
add a comment |
Brilliant answer, Addresses the need of OP
– nehemiah
Nov 12 at 4:23
Brilliant answer, Addresses the need of OP
– nehemiah
Nov 12 at 4:23
Brilliant answer, Addresses the need of OP
– nehemiah
Nov 12 at 4:23
add a comment |
up vote
1
down vote
Thank you Sergey for the succint answer. Here is the decorated version of the same.
import asyncio
import time
def fire_and_forget(f):
from functools import wraps
@wraps(f)
def wrapped(*args, **kwargs):
loop = asyncio.get_event_loop()
if callable(f):
return loop.run_in_executor(None, f, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
return wrapped
@fire_and_forget
def foo():
print("foo() started")
time.sleep(1)
print("foo() completed")
print("Hello")
foo()
print("I didn't wait for foo()")
Produces
>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
add a comment |
up vote
1
down vote
Thank you Sergey for the succint answer. Here is the decorated version of the same.
import asyncio
import time
def fire_and_forget(f):
from functools import wraps
@wraps(f)
def wrapped(*args, **kwargs):
loop = asyncio.get_event_loop()
if callable(f):
return loop.run_in_executor(None, f, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
return wrapped
@fire_and_forget
def foo():
print("foo() started")
time.sleep(1)
print("foo() completed")
print("Hello")
foo()
print("I didn't wait for foo()")
Produces
>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
add a comment |
up vote
1
down vote
up vote
1
down vote
Thank you Sergey for the succint answer. Here is the decorated version of the same.
import asyncio
import time
def fire_and_forget(f):
from functools import wraps
@wraps(f)
def wrapped(*args, **kwargs):
loop = asyncio.get_event_loop()
if callable(f):
return loop.run_in_executor(None, f, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
return wrapped
@fire_and_forget
def foo():
print("foo() started")
time.sleep(1)
print("foo() completed")
print("Hello")
foo()
print("I didn't wait for foo()")
Produces
>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
Thank you Sergey for the succint answer. Here is the decorated version of the same.
import asyncio
import time
def fire_and_forget(f):
from functools import wraps
@wraps(f)
def wrapped(*args, **kwargs):
loop = asyncio.get_event_loop()
if callable(f):
return loop.run_in_executor(None, f, *args, **kwargs)
else:
raise TypeError('Task must be a callable')
return wrapped
@fire_and_forget
def foo():
print("foo() started")
time.sleep(1)
print("foo() completed")
print("Hello")
foo()
print("I didn't wait for foo()")
Produces
>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
edited Dec 7 at 1:17
answered Nov 12 at 4:25
nehemiah
3,85712547
3,85712547
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f37278647%2ffire-and-forget-python-async-await%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Related? stackoverflow.com/q/32808893/1639625 In fact, I think it's a duplicate, but I don't want to instant-dupe-hammer it. Can someone confirm?
– tobias_k
May 20 '16 at 10:00
3
@tobias_k, I don't think it's duplicate. Answer at the link is too broad to be answer for this question.
– Mikhail Gerasimov
May 20 '16 at 11:44
1
Does (1) your "main" process continue running forever ? Or (2) do you want to allow your process to die but allowing forgotten tasks continue their job ? Or (3) do you prefer your main process waiting for forgotten tasks just before ending ?
– Julien Palard
May 26 '16 at 8:23