Limit the number of running jobs in SLURM












6















I am queuing multiple jobs in SLURM. Can I limit the number of parallel running jobs in slurm?



Thanks in advance!






slurm







share|improve this question



























    6















    I am queuing multiple jobs in SLURM. Can I limit the number of parallel running jobs in slurm?



    Thanks in advance!






    slurm







    share|improve this question

























      6












      6








      6


      1






      I am queuing multiple jobs in SLURM. Can I limit the number of parallel running jobs in slurm?



      Thanks in advance!






      slurm







      share|improve this question














      I am queuing multiple jobs in SLURM. Can I limit the number of parallel running jobs in slurm?



      Thanks in advance!






      slurm




      slurm






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Mar 15 '17 at 14:19









      user1447257user1447257

      6111824




      6111824
























          3 Answers
          3






          active

          oldest

          votes


















          7














          If you are not the administrator, your can hold some jobs if you do not want them all to start at the same time, with scontrol hold <JOBID>, and you can delay the submission of some jobs with sbatch --begin=YYYY-MM-DD. Also, if it is a job array, you can limit the number of jobs in the array that are concurrently running with for instance --array=1:100%25 to have 100 jobs in the array but only 25 of them running.






          share|improve this answer
























          • when you hold a job I guess you have to manually unhold it to allow it to run right ? Is it possible to make it automatically unhold whenever a running job ends ?

            – zwlayer
            Apr 18 '18 at 10:13











          • @zwlayer yes you have to unhold them. You can put that at the end of your submission script, or write some script to monitor what is happening, but then I would suggest considering the use of a workflow management tool such as FireWorks for instance.

            – damienfrancois
            Apr 18 '18 at 10:15





















          4














          According to the SLURM Resource Limits documentation, you can limit the total number of jobs that you can run for an association/qos with the MaxJobs parameter. As a reminder, an association is a combination of cluster, account, user name and (optional) partition name.



          You should be able to do something similar to:



          sacctmgr modify user <userid> account=<account_name> set MaxJobs=10


          I found this presentation to be very helpful in case you have more questions.






          share|improve this answer



















          • 1





            I wanted to limit only the number of parallel jobs of a certain step as a user. This is globally.

            – user1447257
            Mar 16 '17 at 9:21



















          0














          According to SLURM documentation, --array=0-15%4 (- sign and not :) will limit the number of simultaneously running tasks from this job array to 4



          I wrote test.sbatch:



          #!/bin/bash
          
# test.sbatch
          
#
          
#SBATCH -J a
          
#SBATCH -p campus
          
#SBATCH -c 1
          
#SBATCH -o %A_%a.output
          

          
mkdir test${SLURM_ARRAY_TASK_ID}
          

          
# sleep for up to 10 minutes to see them running in squeue and 
          
# different times to check that the number of parallel jobs remain constant
          
RANGE=600; number=$RANDOM; let "number %= $RANGE"; echo "$number"
          

          
sleep $number


          and run it with sbatch --array=1-15%4 test.sbatch



          Jobs run as expected (always 4 in parallel) and just create directories and kept running for $number seconds.



          Appreciate comments and suggestions.






          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f42812425%2flimit-the-number-of-running-jobs-in-slurm%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            3 Answers
            3






            active

            oldest

            votes








            3 Answers
            3






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            7














            If you are not the administrator, your can hold some jobs if you do not want them all to start at the same time, with scontrol hold <JOBID>, and you can delay the submission of some jobs with sbatch --begin=YYYY-MM-DD. Also, if it is a job array, you can limit the number of jobs in the array that are concurrently running with for instance --array=1:100%25 to have 100 jobs in the array but only 25 of them running.






            share|improve this answer
























            • when you hold a job I guess you have to manually unhold it to allow it to run right ? Is it possible to make it automatically unhold whenever a running job ends ?

              – zwlayer
              Apr 18 '18 at 10:13











            • @zwlayer yes you have to unhold them. You can put that at the end of your submission script, or write some script to monitor what is happening, but then I would suggest considering the use of a workflow management tool such as FireWorks for instance.

              – damienfrancois
              Apr 18 '18 at 10:15


















            7














            If you are not the administrator, your can hold some jobs if you do not want them all to start at the same time, with scontrol hold <JOBID>, and you can delay the submission of some jobs with sbatch --begin=YYYY-MM-DD. Also, if it is a job array, you can limit the number of jobs in the array that are concurrently running with for instance --array=1:100%25 to have 100 jobs in the array but only 25 of them running.






            share|improve this answer
























            • when you hold a job I guess you have to manually unhold it to allow it to run right ? Is it possible to make it automatically unhold whenever a running job ends ?

              – zwlayer
              Apr 18 '18 at 10:13











            • @zwlayer yes you have to unhold them. You can put that at the end of your submission script, or write some script to monitor what is happening, but then I would suggest considering the use of a workflow management tool such as FireWorks for instance.

              – damienfrancois
              Apr 18 '18 at 10:15
















            7












            7








            7







            If you are not the administrator, your can hold some jobs if you do not want them all to start at the same time, with scontrol hold <JOBID>, and you can delay the submission of some jobs with sbatch --begin=YYYY-MM-DD. Also, if it is a job array, you can limit the number of jobs in the array that are concurrently running with for instance --array=1:100%25 to have 100 jobs in the array but only 25 of them running.






            share|improve this answer













            If you are not the administrator, your can hold some jobs if you do not want them all to start at the same time, with scontrol hold <JOBID>, and you can delay the submission of some jobs with sbatch --begin=YYYY-MM-DD. Also, if it is a job array, you can limit the number of jobs in the array that are concurrently running with for instance --array=1:100%25 to have 100 jobs in the array but only 25 of them running.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Mar 16 '17 at 8:53









            damienfrancoisdamienfrancois

            25.9k54762




            25.9k54762













            • when you hold a job I guess you have to manually unhold it to allow it to run right ? Is it possible to make it automatically unhold whenever a running job ends ?

              – zwlayer
              Apr 18 '18 at 10:13











            • @zwlayer yes you have to unhold them. You can put that at the end of your submission script, or write some script to monitor what is happening, but then I would suggest considering the use of a workflow management tool such as FireWorks for instance.

              – damienfrancois
              Apr 18 '18 at 10:15





















            • when you hold a job I guess you have to manually unhold it to allow it to run right ? Is it possible to make it automatically unhold whenever a running job ends ?

              – zwlayer
              Apr 18 '18 at 10:13











            • @zwlayer yes you have to unhold them. You can put that at the end of your submission script, or write some script to monitor what is happening, but then I would suggest considering the use of a workflow management tool such as FireWorks for instance.

              – damienfrancois
              Apr 18 '18 at 10:15



















            when you hold a job I guess you have to manually unhold it to allow it to run right ? Is it possible to make it automatically unhold whenever a running job ends ?

            – zwlayer
            Apr 18 '18 at 10:13





            when you hold a job I guess you have to manually unhold it to allow it to run right ? Is it possible to make it automatically unhold whenever a running job ends ?

            – zwlayer
            Apr 18 '18 at 10:13













            @zwlayer yes you have to unhold them. You can put that at the end of your submission script, or write some script to monitor what is happening, but then I would suggest considering the use of a workflow management tool such as FireWorks for instance.

            – damienfrancois
            Apr 18 '18 at 10:15







            @zwlayer yes you have to unhold them. You can put that at the end of your submission script, or write some script to monitor what is happening, but then I would suggest considering the use of a workflow management tool such as FireWorks for instance.

            – damienfrancois
            Apr 18 '18 at 10:15















            4














            According to the SLURM Resource Limits documentation, you can limit the total number of jobs that you can run for an association/qos with the MaxJobs parameter. As a reminder, an association is a combination of cluster, account, user name and (optional) partition name.



            You should be able to do something similar to:



            sacctmgr modify user <userid> account=<account_name> set MaxJobs=10


            I found this presentation to be very helpful in case you have more questions.






            share|improve this answer



















            • 1





              I wanted to limit only the number of parallel jobs of a certain step as a user. This is globally.

              – user1447257
              Mar 16 '17 at 9:21
















            4














            According to the SLURM Resource Limits documentation, you can limit the total number of jobs that you can run for an association/qos with the MaxJobs parameter. As a reminder, an association is a combination of cluster, account, user name and (optional) partition name.



            You should be able to do something similar to:



            sacctmgr modify user <userid> account=<account_name> set MaxJobs=10


            I found this presentation to be very helpful in case you have more questions.






            share|improve this answer



















            • 1





              I wanted to limit only the number of parallel jobs of a certain step as a user. This is globally.

              – user1447257
              Mar 16 '17 at 9:21














            4












            4








            4







            According to the SLURM Resource Limits documentation, you can limit the total number of jobs that you can run for an association/qos with the MaxJobs parameter. As a reminder, an association is a combination of cluster, account, user name and (optional) partition name.



            You should be able to do something similar to:



            sacctmgr modify user <userid> account=<account_name> set MaxJobs=10


            I found this presentation to be very helpful in case you have more questions.






            share|improve this answer













            According to the SLURM Resource Limits documentation, you can limit the total number of jobs that you can run for an association/qos with the MaxJobs parameter. As a reminder, an association is a combination of cluster, account, user name and (optional) partition name.



            You should be able to do something similar to:



            sacctmgr modify user <userid> account=<account_name> set MaxJobs=10


            I found this presentation to be very helpful in case you have more questions.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Mar 16 '17 at 1:00









            AndresMAndresM

            962515




            962515








            • 1





              I wanted to limit only the number of parallel jobs of a certain step as a user. This is globally.

              – user1447257
              Mar 16 '17 at 9:21














            • 1





              I wanted to limit only the number of parallel jobs of a certain step as a user. This is globally.

              – user1447257
              Mar 16 '17 at 9:21








            1




            1





            I wanted to limit only the number of parallel jobs of a certain step as a user. This is globally.

            – user1447257
            Mar 16 '17 at 9:21





            I wanted to limit only the number of parallel jobs of a certain step as a user. This is globally.

            – user1447257
            Mar 16 '17 at 9:21











            0














            According to SLURM documentation, --array=0-15%4 (- sign and not :) will limit the number of simultaneously running tasks from this job array to 4



            I wrote test.sbatch:



            #!/bin/bash
            
# test.sbatch
            
#
            
#SBATCH -J a
            
#SBATCH -p campus
            
#SBATCH -c 1
            
#SBATCH -o %A_%a.output
            

            
mkdir test${SLURM_ARRAY_TASK_ID}
            

            
# sleep for up to 10 minutes to see them running in squeue and 
            
# different times to check that the number of parallel jobs remain constant
            
RANGE=600; number=$RANDOM; let "number %= $RANGE"; echo "$number"
            

            
sleep $number


            and run it with sbatch --array=1-15%4 test.sbatch



            Jobs run as expected (always 4 in parallel) and just create directories and kept running for $number seconds.



            Appreciate comments and suggestions.






            share|improve this answer




























              0














              According to SLURM documentation, --array=0-15%4 (- sign and not :) will limit the number of simultaneously running tasks from this job array to 4



              I wrote test.sbatch:



              #!/bin/bash
              
# test.sbatch
              
#
              
#SBATCH -J a
              
#SBATCH -p campus
              
#SBATCH -c 1
              
#SBATCH -o %A_%a.output
              

              
mkdir test${SLURM_ARRAY_TASK_ID}
              

              
# sleep for up to 10 minutes to see them running in squeue and 
              
# different times to check that the number of parallel jobs remain constant
              
RANGE=600; number=$RANDOM; let "number %= $RANGE"; echo "$number"
              

              
sleep $number


              and run it with sbatch --array=1-15%4 test.sbatch



              Jobs run as expected (always 4 in parallel) and just create directories and kept running for $number seconds.



              Appreciate comments and suggestions.






              share|improve this answer


























                0












                0








                0







                According to SLURM documentation, --array=0-15%4 (- sign and not :) will limit the number of simultaneously running tasks from this job array to 4



                I wrote test.sbatch:



                #!/bin/bash
                
# test.sbatch
                
#
                
#SBATCH -J a
                
#SBATCH -p campus
                
#SBATCH -c 1
                
#SBATCH -o %A_%a.output
                

                
mkdir test${SLURM_ARRAY_TASK_ID}
                

                
# sleep for up to 10 minutes to see them running in squeue and 
                
# different times to check that the number of parallel jobs remain constant
                
RANGE=600; number=$RANDOM; let "number %= $RANGE"; echo "$number"
                

                
sleep $number


                and run it with sbatch --array=1-15%4 test.sbatch



                Jobs run as expected (always 4 in parallel) and just create directories and kept running for $number seconds.



                Appreciate comments and suggestions.






                share|improve this answer













                According to SLURM documentation, --array=0-15%4 (- sign and not :) will limit the number of simultaneously running tasks from this job array to 4



                I wrote test.sbatch:



                #!/bin/bash
                
# test.sbatch
                
#
                
#SBATCH -J a
                
#SBATCH -p campus
                
#SBATCH -c 1
                
#SBATCH -o %A_%a.output
                

                
mkdir test${SLURM_ARRAY_TASK_ID}
                

                
# sleep for up to 10 minutes to see them running in squeue and 
                
# different times to check that the number of parallel jobs remain constant
                
RANGE=600; number=$RANDOM; let "number %= $RANGE"; echo "$number"
                

                
sleep $number


                and run it with sbatch --array=1-15%4 test.sbatch



                Jobs run as expected (always 4 in parallel) and just create directories and kept running for $number seconds.



                Appreciate comments and suggestions.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Jan 19 at 21:39









                aerijmanaerijman

                436




                436






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f42812425%2flimit-the-number-of-running-jobs-in-slurm%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    Bressuire

                    Image
                    Subprefecture and commune in Nouvelle-Aquitaine, France Bressuire Subprefecture and commune Chateau de Bressuire and the Eglise Notre-Dame Coat of arms Location of Bressuire Bressuire Show map of France Bressuire Show map of Nouvelle-Aquitaine Coordinates: 46°50′27″N 0°29′14″W  /  46.8408°N 0.4872°W  / 46.8408; -0.4872 Coordinates: 46°50′27″N 0°29′14″W  /  46.8408°N 0.4872°W  / 46.8408; -0.4872 Country France Region Nouvelle-Aquitaine Department Deux-Sèvres Arrondissement Bressuire Canton Bressuire Government  • Mayor .mw-parser-output .nobold{font-weight:normal} (2014–20) Jean Michel Bernier Area 1 180.59 km 2 (69.73 sq mi) Population (2014) 2 19,300  • Density 110/km 2 (280/sq mi) Time zone UTC+01:00 (CET)  • Summer (DST) UTC+02:00 (CEST) INSEE/Postal code 79049 /79300 Elevation 98–236 m (322–774 ft) (avg. 173 m or 568 ft) 1 French Land Register data, which exclude...
                    Read more

                    Vorschmack

                    Image
                    Vorschmack Ukrainian Jewish-style vorschmack served on rye bread Course Hors d'oeuvre Region or state Eastern Europe Associated national cuisine Ashkenazi Jewish, Finnish, German, Ukrainian, Polish, Russian Main ingredients Ground meat and/or fish Cookbook: Vorschmack   Media: Vorschmack Vorschmack or forshmak (Yiddish: פֿאָרשמאַק ‎, from archaic German Vorschmack , "foretaste" [1] or "appetizer" [2] ) is an originally East European dish made of salty minced fish or meat. Different variants of this dish are especially common in Ashkenazi Jewish and Finnish cuisine. Some varieties are also known in Russian and Polish cuisine. Contents 1 In Jewish cuisine 2 In Russian cuisine 3 In Polish cuisine 4 In Finnish cuisine 5 See also 6 References In Jewish cuisine According to Gil Marks, the German name points to the possible Germanic origin of this dish. [1] William Pokhlyobkin descr...
                    Read more

                    Quarantine

                    Image
                    For other uses, see Quarantine (disambiguation). Signal flag "Lima" called the "Yellow Jack" which when flown in harbor means ship is under quarantine. A simple yellow flag (also called the "Yellow Jack") had historically been used to signal quarantine (it stands for Q among signal flags), but now indicates the opposite, as a signal of a ship free of disease that requests boarding and inspection. A quarantine is used to separate and restrict the movement of people; it is 'a restraint upon the activities or communication of persons or the transport of goods designed to prevent the spread of disease or pests', for a certain period of time. [1] This is often used in connection to disease and illness, such as those who may possibly have been exposed to a communicable disease. [2] The term is often erroneously used to mean medical isolation, which is "to separate ill persons who have a communicable disease from those who are healthy...
                    Read more