Airflow: How to unpause DAG from python script
I'm creating a python script to generate DAGs (generate a new python file with the DAG specifications) from templates. It all works fine, except that I need the DAG to be be generated as unpaused.
I've searched and tried to run shell commands in the script like this:
bash_command1 = 'airflow list_dags'
bash_command2 = 'airflow trigger_dag ' + str(DAG_ID)
bash_command3 = 'airflow list_tasks ' + str(DAG_ID)
bash_command4 = 'airflow unpause '+ str(DAG_ID)
subprocess.call(bash_command1.split())
subprocess.call(bash_command2.split())
subprocess.call(bash_command3.split())
subprocess.call(bash_command4.split())
But every time I create a new DAG it is shown as paused in the web UI.
By the research I´ve made, the command airflow unpause <dag_id> should solve the problem, but when the script executes it, I get the error:
Traceback (most recent call last):
File "/home/cubo/anaconda2/bin/airflow", line 28, in <module>
args.func(args)
File "/home/cubo/anaconda2/lib/python2.7/site-packages/airflow/bin/cli.py", line 303, in unpause
set_is_paused(False, args, dag)
File "/home/cubo/anaconda2/lib/python2.7/site- packages/airflow/bin/cli.py", line 312, in set_is_paused
dm.is_paused = is_paused
AttributeError: 'NoneType' object has no attribute 'is_paused'
But when I execute the same airflow unpause <dag_id> command in the terminal it works fine, and it prints:
Dag: <DAG: DAG_ID>, paused: False
Any help would be greatly appreciated.
python-2.7 airflow
asked Nov 16 '18 at 1:59
David FDavid F
33
add a comment |
I'm creating a python script to generate DAGs (generate a new python file with the DAG specifications) from templates. It all works fine, except that I need the DAG to be be generated as unpaused.
I've searched and tried to run shell commands in the script like this:
bash_command1 = 'airflow list_dags'
bash_command2 = 'airflow trigger_dag ' + str(DAG_ID)
bash_command3 = 'airflow list_tasks ' + str(DAG_ID)
bash_command4 = 'airflow unpause '+ str(DAG_ID)
subprocess.call(bash_command1.split())
subprocess.call(bash_command2.split())
subprocess.call(bash_command3.split())
subprocess.call(bash_command4.split())
But every time I create a new DAG it is shown as paused in the web UI.
By the research I´ve made, the command airflow unpause <dag_id> should solve the problem, but when the script executes it, I get the error:
Traceback (most recent call last):
File "/home/cubo/anaconda2/bin/airflow", line 28, in <module>
args.func(args)
File "/home/cubo/anaconda2/lib/python2.7/site-packages/airflow/bin/cli.py", line 303, in unpause
set_is_paused(False, args, dag)
File "/home/cubo/anaconda2/lib/python2.7/site- packages/airflow/bin/cli.py", line 312, in set_is_paused
dm.is_paused = is_paused
AttributeError: 'NoneType' object has no attribute 'is_paused'
But when I execute the same airflow unpause <dag_id> command in the terminal it works fine, and it prints:
Dag: <DAG: DAG_ID>, paused: False
Any help would be greatly appreciated.
python-2.7 airflow
asked Nov 16 '18 at 1:59
David FDavid F
33
add a comment |
I'm creating a python script to generate DAGs (generate a new python file with the DAG specifications) from templates. It all works fine, except that I need the DAG to be be generated as unpaused.
I've searched and tried to run shell commands in the script like this:
bash_command1 = 'airflow list_dags'
bash_command2 = 'airflow trigger_dag ' + str(DAG_ID)
bash_command3 = 'airflow list_tasks ' + str(DAG_ID)
bash_command4 = 'airflow unpause '+ str(DAG_ID)
subprocess.call(bash_command1.split())
subprocess.call(bash_command2.split())
subprocess.call(bash_command3.split())
subprocess.call(bash_command4.split())
But every time I create a new DAG it is shown as paused in the web UI.
By the research I´ve made, the command airflow unpause <dag_id> should solve the problem, but when the script executes it, I get the error:
Traceback (most recent call last):
File "/home/cubo/anaconda2/bin/airflow", line 28, in <module>
args.func(args)
File "/home/cubo/anaconda2/lib/python2.7/site-packages/airflow/bin/cli.py", line 303, in unpause
set_is_paused(False, args, dag)
File "/home/cubo/anaconda2/lib/python2.7/site- packages/airflow/bin/cli.py", line 312, in set_is_paused
dm.is_paused = is_paused
AttributeError: 'NoneType' object has no attribute 'is_paused'
But when I execute the same airflow unpause <dag_id> command in the terminal it works fine, and it prints:
Dag: <DAG: DAG_ID>, paused: False
Any help would be greatly appreciated.
python-2.7 airflow
asked Nov 16 '18 at 1:59
David FDavid F
33
I'm creating a python script to generate DAGs (generate a new python file with the DAG specifications) from templates. It all works fine, except that I need the DAG to be be generated as unpaused.
I've searched and tried to run shell commands in the script like this:
bash_command1 = 'airflow list_dags'
bash_command2 = 'airflow trigger_dag ' + str(DAG_ID)
bash_command3 = 'airflow list_tasks ' + str(DAG_ID)
bash_command4 = 'airflow unpause '+ str(DAG_ID)
subprocess.call(bash_command1.split())
subprocess.call(bash_command2.split())
subprocess.call(bash_command3.split())
subprocess.call(bash_command4.split())
But every time I create a new DAG it is shown as paused in the web UI.
By the research I´ve made, the command airflow unpause <dag_id> should solve the problem, but when the script executes it, I get the error:
Traceback (most recent call last):
File "/home/cubo/anaconda2/bin/airflow", line 28, in <module>
args.func(args)
File "/home/cubo/anaconda2/lib/python2.7/site-packages/airflow/bin/cli.py", line 303, in unpause
set_is_paused(False, args, dag)
File "/home/cubo/anaconda2/lib/python2.7/site- packages/airflow/bin/cli.py", line 312, in set_is_paused
dm.is_paused = is_paused
AttributeError: 'NoneType' object has no attribute 'is_paused'
But when I execute the same airflow unpause <dag_id> command in the terminal it works fine, and it prints:
Dag: <DAG: DAG_ID>, paused: False
Any help would be greatly appreciated.
python-2.7 airflow
python-2.7 airflow
asked Nov 16 '18 at 1:59
David FDavid F
33
asked Nov 16 '18 at 1:59
David FDavid F
33
asked Nov 16 '18 at 1:59
David FDavid F
33
asked Nov 16 '18 at 1:59
David FDavid F
33
asked Nov 16 '18 at 1:59
David FDavid F
33
33
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Airflow (1.8 and newer) pauses new dags by default. If you want all DAGs to be unpaused at creation, you can override the Airflow config to retain the prior behavior of unpausing at creation.
Here's the link that walks you through setting configuration options. You want to set a core configuration setting: dags_are_paused_at_creation to False.
We use the environment variable approach on my team.
answered Nov 16 '18 at 2:16
TeresaTeresa
756
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53330414%2fairflow-how-to-unpause-dag-from-python-script%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Airflow (1.8 and newer) pauses new dags by default. If you want all DAGs to be unpaused at creation, you can override the Airflow config to retain the prior behavior of unpausing at creation.
Here's the link that walks you through setting configuration options. You want to set a core configuration setting: dags_are_paused_at_creation to False.
We use the environment variable approach on my team.
answered Nov 16 '18 at 2:16
TeresaTeresa
756
add a comment |
Airflow (1.8 and newer) pauses new dags by default. If you want all DAGs to be unpaused at creation, you can override the Airflow config to retain the prior behavior of unpausing at creation.
Here's the link that walks you through setting configuration options. You want to set a core configuration setting: dags_are_paused_at_creation to False.
We use the environment variable approach on my team.
answered Nov 16 '18 at 2:16
TeresaTeresa
756
add a comment |
Airflow (1.8 and newer) pauses new dags by default. If you want all DAGs to be unpaused at creation, you can override the Airflow config to retain the prior behavior of unpausing at creation.
Here's the link that walks you through setting configuration options. You want to set a core configuration setting: dags_are_paused_at_creation to False.
We use the environment variable approach on my team.
answered Nov 16 '18 at 2:16
TeresaTeresa
756
Airflow (1.8 and newer) pauses new dags by default. If you want all DAGs to be unpaused at creation, you can override the Airflow config to retain the prior behavior of unpausing at creation.
Here's the link that walks you through setting configuration options. You want to set a core configuration setting: dags_are_paused_at_creation to False.
We use the environment variable approach on my team.
answered Nov 16 '18 at 2:16
TeresaTeresa
756
edited Nov 16 '18 at 2:21
answered Nov 16 '18 at 2:16
TeresaTeresa
756
answered Nov 16 '18 at 2:16
TeresaTeresa
756
answered Nov 16 '18 at 2:16
TeresaTeresa
756
756
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53330414%2fairflow-how-to-unpause-dag-from-python-script%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
