When using Powershell Jobs, Runspaces, or Workflows, are the threads being executed on separate cores?
When using Powershell Jobs, Runspaces, or Workflows, are the threads being executed on separate cores? (and if so, how do we tell powershell how many cores to use? -- sorry that's 2 questions.)
.Net has the Task Parallel Library, which allows a 'for loop' to run in parallel, using all available cores (here is one example). Does Powershell Jobs, Runspaces, or Workflows do something similar? And by similar, I mean are the threads actually running on separate cores, in parallel?
I found a similar question here, but it seems unclear (to me anyway) if the threads are executed on separate cores. It seems that sometimes multi-threads are mistaken for parallel, as mentioned here.
If using multiple cores with Powershell is not possible, I will use C# or perhaps Python, but my first choice is to use Powershell because (insert long list of reasons).
If it's relevant, I'm trying to do all this to help a co-worker who does server admin type stuff. Currently, his powershell script loops through a list of servers, and foreach server, does stuff. Currently the script runs on an 8 core machine, but is only using one core. I'm sure there are other ways to boost performance, but doing it in parallel is my current goal.
powershell parallel-processing
add a comment |
When using Powershell Jobs, Runspaces, or Workflows, are the threads being executed on separate cores? (and if so, how do we tell powershell how many cores to use? -- sorry that's 2 questions.)
.Net has the Task Parallel Library, which allows a 'for loop' to run in parallel, using all available cores (here is one example). Does Powershell Jobs, Runspaces, or Workflows do something similar? And by similar, I mean are the threads actually running on separate cores, in parallel?
I found a similar question here, but it seems unclear (to me anyway) if the threads are executed on separate cores. It seems that sometimes multi-threads are mistaken for parallel, as mentioned here.
If using multiple cores with Powershell is not possible, I will use C# or perhaps Python, but my first choice is to use Powershell because (insert long list of reasons).
If it's relevant, I'm trying to do all this to help a co-worker who does server admin type stuff. Currently, his powershell script loops through a list of servers, and foreach server, does stuff. Currently the script runs on an 8 core machine, but is only using one core. I'm sure there are other ways to boost performance, but doing it in parallel is my current goal.
powershell parallel-processing
Have a look forInvoke-All
. Someone has put what you are after into a nice little function.
– Drew
Nov 16 '18 at 4:55
or Invoke-Parallel github.com/RamblingCookieMonster/Invoke-Parallel
– Owain Esau
Nov 16 '18 at 4:57
add a comment |
When using Powershell Jobs, Runspaces, or Workflows, are the threads being executed on separate cores? (and if so, how do we tell powershell how many cores to use? -- sorry that's 2 questions.)
.Net has the Task Parallel Library, which allows a 'for loop' to run in parallel, using all available cores (here is one example). Does Powershell Jobs, Runspaces, or Workflows do something similar? And by similar, I mean are the threads actually running on separate cores, in parallel?
I found a similar question here, but it seems unclear (to me anyway) if the threads are executed on separate cores. It seems that sometimes multi-threads are mistaken for parallel, as mentioned here.
If using multiple cores with Powershell is not possible, I will use C# or perhaps Python, but my first choice is to use Powershell because (insert long list of reasons).
If it's relevant, I'm trying to do all this to help a co-worker who does server admin type stuff. Currently, his powershell script loops through a list of servers, and foreach server, does stuff. Currently the script runs on an 8 core machine, but is only using one core. I'm sure there are other ways to boost performance, but doing it in parallel is my current goal.
powershell parallel-processing
When using Powershell Jobs, Runspaces, or Workflows, are the threads being executed on separate cores? (and if so, how do we tell powershell how many cores to use? -- sorry that's 2 questions.)
.Net has the Task Parallel Library, which allows a 'for loop' to run in parallel, using all available cores (here is one example). Does Powershell Jobs, Runspaces, or Workflows do something similar? And by similar, I mean are the threads actually running on separate cores, in parallel?
I found a similar question here, but it seems unclear (to me anyway) if the threads are executed on separate cores. It seems that sometimes multi-threads are mistaken for parallel, as mentioned here.
If using multiple cores with Powershell is not possible, I will use C# or perhaps Python, but my first choice is to use Powershell because (insert long list of reasons).
If it's relevant, I'm trying to do all this to help a co-worker who does server admin type stuff. Currently, his powershell script loops through a list of servers, and foreach server, does stuff. Currently the script runs on an 8 core machine, but is only using one core. I'm sure there are other ways to boost performance, but doing it in parallel is my current goal.
powershell parallel-processing
powershell parallel-processing
asked Nov 16 '18 at 3:45
greggreg
87112
87112
Have a look forInvoke-All
. Someone has put what you are after into a nice little function.
– Drew
Nov 16 '18 at 4:55
or Invoke-Parallel github.com/RamblingCookieMonster/Invoke-Parallel
– Owain Esau
Nov 16 '18 at 4:57
add a comment |
Have a look forInvoke-All
. Someone has put what you are after into a nice little function.
– Drew
Nov 16 '18 at 4:55
or Invoke-Parallel github.com/RamblingCookieMonster/Invoke-Parallel
– Owain Esau
Nov 16 '18 at 4:57
Have a look for
Invoke-All
. Someone has put what you are after into a nice little function.– Drew
Nov 16 '18 at 4:55
Have a look for
Invoke-All
. Someone has put what you are after into a nice little function.– Drew
Nov 16 '18 at 4:55
or Invoke-Parallel github.com/RamblingCookieMonster/Invoke-Parallel
– Owain Esau
Nov 16 '18 at 4:57
or Invoke-Parallel github.com/RamblingCookieMonster/Invoke-Parallel
– Owain Esau
Nov 16 '18 at 4:57
add a comment |
1 Answer
1
active
oldest
votes
I would say yes if by separate cores, you mean logical cores (not physical cores). Run something like this and notice the output from “$num”. $num represents the current logical (not physical) core on which a thread is running. If this script runs on a dual core machine (with 4 logical cores), the output of $num is a 0, 1, 2, or 3. See here for a better explanation on Logical vs Physical CPU.
$sb = {
param($sbarg)
$MethodDefinition = @'
[DllImport("kernel32.dll", CharSet = CharSet.Unicode)]
public static extern int GetCurrentProcessorNumber();
'@
$Kernel32 = Add-Type -MemberDefinition $MethodDefinition -Name 'Kernel32' -Namespace 'Win32' -PassThru
0..10 | % {
$num = $Kernel32::GetCurrentProcessorNumber()
Write-Output "[$sbarg]:[$_] on '$num'"
# simulate some work that may make the cpu busy
# perhaps watch in task manager as well to see load
$result = 1; foreach ($number in 1..1000000) {$result = $result * $number};
Start-Sleep -Milliseconds 500
}
}
0..10 | % {
Start-Job -ScriptBlock $sb -ArgumentList $_
}
Get-Job
# Wait for it all to complete
While (Get-Job -State "Running")
{
Write-Output "waiting..."
Start-Sleep 2
}
# Getting the information back from the jobs
Get-Job | Receive-Job
# Clean Up
Get-Job | Remove-Job
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53331116%2fwhen-using-powershell-jobs-runspaces-or-workflows-are-the-threads-being-execu%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I would say yes if by separate cores, you mean logical cores (not physical cores). Run something like this and notice the output from “$num”. $num represents the current logical (not physical) core on which a thread is running. If this script runs on a dual core machine (with 4 logical cores), the output of $num is a 0, 1, 2, or 3. See here for a better explanation on Logical vs Physical CPU.
$sb = {
param($sbarg)
$MethodDefinition = @'
[DllImport("kernel32.dll", CharSet = CharSet.Unicode)]
public static extern int GetCurrentProcessorNumber();
'@
$Kernel32 = Add-Type -MemberDefinition $MethodDefinition -Name 'Kernel32' -Namespace 'Win32' -PassThru
0..10 | % {
$num = $Kernel32::GetCurrentProcessorNumber()
Write-Output "[$sbarg]:[$_] on '$num'"
# simulate some work that may make the cpu busy
# perhaps watch in task manager as well to see load
$result = 1; foreach ($number in 1..1000000) {$result = $result * $number};
Start-Sleep -Milliseconds 500
}
}
0..10 | % {
Start-Job -ScriptBlock $sb -ArgumentList $_
}
Get-Job
# Wait for it all to complete
While (Get-Job -State "Running")
{
Write-Output "waiting..."
Start-Sleep 2
}
# Getting the information back from the jobs
Get-Job | Receive-Job
# Clean Up
Get-Job | Remove-Job
add a comment |
I would say yes if by separate cores, you mean logical cores (not physical cores). Run something like this and notice the output from “$num”. $num represents the current logical (not physical) core on which a thread is running. If this script runs on a dual core machine (with 4 logical cores), the output of $num is a 0, 1, 2, or 3. See here for a better explanation on Logical vs Physical CPU.
$sb = {
param($sbarg)
$MethodDefinition = @'
[DllImport("kernel32.dll", CharSet = CharSet.Unicode)]
public static extern int GetCurrentProcessorNumber();
'@
$Kernel32 = Add-Type -MemberDefinition $MethodDefinition -Name 'Kernel32' -Namespace 'Win32' -PassThru
0..10 | % {
$num = $Kernel32::GetCurrentProcessorNumber()
Write-Output "[$sbarg]:[$_] on '$num'"
# simulate some work that may make the cpu busy
# perhaps watch in task manager as well to see load
$result = 1; foreach ($number in 1..1000000) {$result = $result * $number};
Start-Sleep -Milliseconds 500
}
}
0..10 | % {
Start-Job -ScriptBlock $sb -ArgumentList $_
}
Get-Job
# Wait for it all to complete
While (Get-Job -State "Running")
{
Write-Output "waiting..."
Start-Sleep 2
}
# Getting the information back from the jobs
Get-Job | Receive-Job
# Clean Up
Get-Job | Remove-Job
add a comment |
I would say yes if by separate cores, you mean logical cores (not physical cores). Run something like this and notice the output from “$num”. $num represents the current logical (not physical) core on which a thread is running. If this script runs on a dual core machine (with 4 logical cores), the output of $num is a 0, 1, 2, or 3. See here for a better explanation on Logical vs Physical CPU.
$sb = {
param($sbarg)
$MethodDefinition = @'
[DllImport("kernel32.dll", CharSet = CharSet.Unicode)]
public static extern int GetCurrentProcessorNumber();
'@
$Kernel32 = Add-Type -MemberDefinition $MethodDefinition -Name 'Kernel32' -Namespace 'Win32' -PassThru
0..10 | % {
$num = $Kernel32::GetCurrentProcessorNumber()
Write-Output "[$sbarg]:[$_] on '$num'"
# simulate some work that may make the cpu busy
# perhaps watch in task manager as well to see load
$result = 1; foreach ($number in 1..1000000) {$result = $result * $number};
Start-Sleep -Milliseconds 500
}
}
0..10 | % {
Start-Job -ScriptBlock $sb -ArgumentList $_
}
Get-Job
# Wait for it all to complete
While (Get-Job -State "Running")
{
Write-Output "waiting..."
Start-Sleep 2
}
# Getting the information back from the jobs
Get-Job | Receive-Job
# Clean Up
Get-Job | Remove-Job
I would say yes if by separate cores, you mean logical cores (not physical cores). Run something like this and notice the output from “$num”. $num represents the current logical (not physical) core on which a thread is running. If this script runs on a dual core machine (with 4 logical cores), the output of $num is a 0, 1, 2, or 3. See here for a better explanation on Logical vs Physical CPU.
$sb = {
param($sbarg)
$MethodDefinition = @'
[DllImport("kernel32.dll", CharSet = CharSet.Unicode)]
public static extern int GetCurrentProcessorNumber();
'@
$Kernel32 = Add-Type -MemberDefinition $MethodDefinition -Name 'Kernel32' -Namespace 'Win32' -PassThru
0..10 | % {
$num = $Kernel32::GetCurrentProcessorNumber()
Write-Output "[$sbarg]:[$_] on '$num'"
# simulate some work that may make the cpu busy
# perhaps watch in task manager as well to see load
$result = 1; foreach ($number in 1..1000000) {$result = $result * $number};
Start-Sleep -Milliseconds 500
}
}
0..10 | % {
Start-Job -ScriptBlock $sb -ArgumentList $_
}
Get-Job
# Wait for it all to complete
While (Get-Job -State "Running")
{
Write-Output "waiting..."
Start-Sleep 2
}
# Getting the information back from the jobs
Get-Job | Receive-Job
# Clean Up
Get-Job | Remove-Job
edited Nov 19 '18 at 7:46
greg
87112
87112
answered Nov 16 '18 at 6:00
Kory GillKory Gill
5,49611026
5,49611026
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53331116%2fwhen-using-powershell-jobs-runspaces-or-workflows-are-the-threads-being-execu%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Have a look for
Invoke-All
. Someone has put what you are after into a nice little function.– Drew
Nov 16 '18 at 4:55
or Invoke-Parallel github.com/RamblingCookieMonster/Invoke-Parallel
– Owain Esau
Nov 16 '18 at 4:57