Multi level lock?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
Playing around with parallel processing, I have a Parallel.For loop that does a lookup on an enumerable, and if it doesn't find what it's looking for, adds it. I keep running into issues where one thread modifies the enumerable while another thread is doing a lookup, though, which makes it throw an exception. The obvious solution is to use lock on both the lookup and the entry, but that's a bit brute force. I'd like it to be able to run multiple concurrent lookups, so I don't want to completely lock it while one thread is doing a lookup, but I do want to bar it from doing an add while a lookup is in progress. Is there any way of doing a two-level lock like that?
c# multithreading parallel-processing locking
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
add a comment |
Playing around with parallel processing, I have a Parallel.For loop that does a lookup on an enumerable, and if it doesn't find what it's looking for, adds it. I keep running into issues where one thread modifies the enumerable while another thread is doing a lookup, though, which makes it throw an exception. The obvious solution is to use lock on both the lookup and the entry, but that's a bit brute force. I'd like it to be able to run multiple concurrent lookups, so I don't want to completely lock it while one thread is doing a lookup, but I do want to bar it from doing an add while a lookup is in progress. Is there any way of doing a two-level lock like that?
c# multithreading parallel-processing locking
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
1
Consider using one of the thread safe collections inSystem.Collections.Concurrentinstead.
– juharr
Nov 16 '18 at 13:28
What is the relation of # of writes to # of reads ? Are they coarsely the same? Far more Reads than Writes? Far more Writes than Reads? Efficient solutions may depend on that.
– Fildor
Nov 16 '18 at 13:30
More reads than writes, but coarsely the same. I'm using the SharePoint.Client library, iterating through a number of folder names, checking whether each folder exists as a subfolder in SharePoint, and creating it if it doesn't.
– Tam Coton
Nov 16 '18 at 13:34
add a comment |
Playing around with parallel processing, I have a Parallel.For loop that does a lookup on an enumerable, and if it doesn't find what it's looking for, adds it. I keep running into issues where one thread modifies the enumerable while another thread is doing a lookup, though, which makes it throw an exception. The obvious solution is to use lock on both the lookup and the entry, but that's a bit brute force. I'd like it to be able to run multiple concurrent lookups, so I don't want to completely lock it while one thread is doing a lookup, but I do want to bar it from doing an add while a lookup is in progress. Is there any way of doing a two-level lock like that?
c# multithreading parallel-processing locking
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
Playing around with parallel processing, I have a Parallel.For loop that does a lookup on an enumerable, and if it doesn't find what it's looking for, adds it. I keep running into issues where one thread modifies the enumerable while another thread is doing a lookup, though, which makes it throw an exception. The obvious solution is to use lock on both the lookup and the entry, but that's a bit brute force. I'd like it to be able to run multiple concurrent lookups, so I don't want to completely lock it while one thread is doing a lookup, but I do want to bar it from doing an add while a lookup is in progress. Is there any way of doing a two-level lock like that?
c# multithreading parallel-processing locking
c# multithreading parallel-processing locking
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
asked Nov 16 '18 at 13:18
Tam CotonTam Coton
339316
339316
1
Consider using one of the thread safe collections inSystem.Collections.Concurrentinstead.
– juharr
Nov 16 '18 at 13:28
What is the relation of # of writes to # of reads ? Are they coarsely the same? Far more Reads than Writes? Far more Writes than Reads? Efficient solutions may depend on that.
– Fildor
Nov 16 '18 at 13:30
More reads than writes, but coarsely the same. I'm using the SharePoint.Client library, iterating through a number of folder names, checking whether each folder exists as a subfolder in SharePoint, and creating it if it doesn't.
– Tam Coton
Nov 16 '18 at 13:34
add a comment |
1
Consider using one of the thread safe collections inSystem.Collections.Concurrentinstead.
– juharr
Nov 16 '18 at 13:28
What is the relation of # of writes to # of reads ? Are they coarsely the same? Far more Reads than Writes? Far more Writes than Reads? Efficient solutions may depend on that.
– Fildor
Nov 16 '18 at 13:30
More reads than writes, but coarsely the same. I'm using the SharePoint.Client library, iterating through a number of folder names, checking whether each folder exists as a subfolder in SharePoint, and creating it if it doesn't.
– Tam Coton
Nov 16 '18 at 13:34
1
1
Consider using one of the thread safe collections in
System.Collections.Concurrent instead.– juharr
Nov 16 '18 at 13:28
Consider using one of the thread safe collections in
System.Collections.Concurrent instead.– juharr
Nov 16 '18 at 13:28
What is the relation of # of writes to # of reads ? Are they coarsely the same? Far more Reads than Writes? Far more Writes than Reads? Efficient solutions may depend on that.
– Fildor
Nov 16 '18 at 13:30
What is the relation of # of writes to # of reads ? Are they coarsely the same? Far more Reads than Writes? Far more Writes than Reads? Efficient solutions may depend on that.
– Fildor
Nov 16 '18 at 13:30
More reads than writes, but coarsely the same. I'm using the SharePoint.Client library, iterating through a number of folder names, checking whether each folder exists as a subfolder in SharePoint, and creating it if it doesn't.
– Tam Coton
Nov 16 '18 at 13:34
More reads than writes, but coarsely the same. I'm using the SharePoint.Client library, iterating through a number of folder names, checking whether each folder exists as a subfolder in SharePoint, and creating it if it doesn't.
– Tam Coton
Nov 16 '18 at 13:34
add a comment |
2 Answers
2
active
oldest
votes
You may want to use a ReaderWriterLock.
private ReaderWriterLockSlim lock = new ReaderWriterLockSlim();
public string Read () {
lock.EnterReadLock ();
try {
return "xxx";
} finally {
lock.ExitReadLock();
}
}
public void Write () {
lock.EnterWriteLock ();
try {
// ...
} finally {
lock.ExiteWriteLock ();
}
}
Multiple threads may own the "reader" part of a ReaderWriterLock, but only one can own the "writer" part (and there can be no reader while there is a writer).
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
This looks like exactly what I was after! Many thanks!
– Tam Coton
Nov 16 '18 at 13:54
add a comment |
I'd be thinking of using some form of immutable collection and Interlocked to replace the shared instance.
E.g. you mention "Look ups" so might an ImmutableDictionary be a good match?
Then for the lookup and update, you have something like:
var local = sharedReference;
while(!local.ContainsKey(lookupValue))
{
var newValue = /* Whatever */
var newLocal = local.Add(lookupValue,newValue);
var result = Interlocked.CompareExchange(ref sharedReference, newLocal, local);
if(result == local) break;
local = result;
}
//At this point, local definitely contains a key for lookupValue
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53338703%2fmulti-level-lock%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You may want to use a ReaderWriterLock.
private ReaderWriterLockSlim lock = new ReaderWriterLockSlim();
public string Read () {
lock.EnterReadLock ();
try {
return "xxx";
} finally {
lock.ExitReadLock();
}
}
public void Write () {
lock.EnterWriteLock ();
try {
// ...
} finally {
lock.ExiteWriteLock ();
}
}
Multiple threads may own the "reader" part of a ReaderWriterLock, but only one can own the "writer" part (and there can be no reader while there is a writer).
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
This looks like exactly what I was after! Many thanks!
– Tam Coton
Nov 16 '18 at 13:54
add a comment |
You may want to use a ReaderWriterLock.
private ReaderWriterLockSlim lock = new ReaderWriterLockSlim();
public string Read () {
lock.EnterReadLock ();
try {
return "xxx";
} finally {
lock.ExitReadLock();
}
}
public void Write () {
lock.EnterWriteLock ();
try {
// ...
} finally {
lock.ExiteWriteLock ();
}
}
Multiple threads may own the "reader" part of a ReaderWriterLock, but only one can own the "writer" part (and there can be no reader while there is a writer).
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
This looks like exactly what I was after! Many thanks!
– Tam Coton
Nov 16 '18 at 13:54
add a comment |
You may want to use a ReaderWriterLock.
private ReaderWriterLockSlim lock = new ReaderWriterLockSlim();
public string Read () {
lock.EnterReadLock ();
try {
return "xxx";
} finally {
lock.ExitReadLock();
}
}
public void Write () {
lock.EnterWriteLock ();
try {
// ...
} finally {
lock.ExiteWriteLock ();
}
}
Multiple threads may own the "reader" part of a ReaderWriterLock, but only one can own the "writer" part (and there can be no reader while there is a writer).
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
You may want to use a ReaderWriterLock.
private ReaderWriterLockSlim lock = new ReaderWriterLockSlim();
public string Read () {
lock.EnterReadLock ();
try {
return "xxx";
} finally {
lock.ExitReadLock();
}
}
public void Write () {
lock.EnterWriteLock ();
try {
// ...
} finally {
lock.ExiteWriteLock ();
}
}
Multiple threads may own the "reader" part of a ReaderWriterLock, but only one can own the "writer" part (and there can be no reader while there is a writer).
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
answered Nov 16 '18 at 13:29
Etienne MiretEtienne Miret
4,78021729
4,78021729
This looks like exactly what I was after! Many thanks!
– Tam Coton
Nov 16 '18 at 13:54
add a comment |
This looks like exactly what I was after! Many thanks!
– Tam Coton
Nov 16 '18 at 13:54
This looks like exactly what I was after! Many thanks!
– Tam Coton
Nov 16 '18 at 13:54
This looks like exactly what I was after! Many thanks!
– Tam Coton
Nov 16 '18 at 13:54
add a comment |
I'd be thinking of using some form of immutable collection and Interlocked to replace the shared instance.
E.g. you mention "Look ups" so might an ImmutableDictionary be a good match?
Then for the lookup and update, you have something like:
var local = sharedReference;
while(!local.ContainsKey(lookupValue))
{
var newValue = /* Whatever */
var newLocal = local.Add(lookupValue,newValue);
var result = Interlocked.CompareExchange(ref sharedReference, newLocal, local);
if(result == local) break;
local = result;
}
//At this point, local definitely contains a key for lookupValue
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
add a comment |
I'd be thinking of using some form of immutable collection and Interlocked to replace the shared instance.
E.g. you mention "Look ups" so might an ImmutableDictionary be a good match?
Then for the lookup and update, you have something like:
var local = sharedReference;
while(!local.ContainsKey(lookupValue))
{
var newValue = /* Whatever */
var newLocal = local.Add(lookupValue,newValue);
var result = Interlocked.CompareExchange(ref sharedReference, newLocal, local);
if(result == local) break;
local = result;
}
//At this point, local definitely contains a key for lookupValue
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
add a comment |
I'd be thinking of using some form of immutable collection and Interlocked to replace the shared instance.
E.g. you mention "Look ups" so might an ImmutableDictionary be a good match?
Then for the lookup and update, you have something like:
var local = sharedReference;
while(!local.ContainsKey(lookupValue))
{
var newValue = /* Whatever */
var newLocal = local.Add(lookupValue,newValue);
var result = Interlocked.CompareExchange(ref sharedReference, newLocal, local);
if(result == local) break;
local = result;
}
//At this point, local definitely contains a key for lookupValue
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
I'd be thinking of using some form of immutable collection and Interlocked to replace the shared instance.
E.g. you mention "Look ups" so might an ImmutableDictionary be a good match?
Then for the lookup and update, you have something like:
var local = sharedReference;
while(!local.ContainsKey(lookupValue))
{
var newValue = /* Whatever */
var newLocal = local.Add(lookupValue,newValue);
var result = Interlocked.CompareExchange(ref sharedReference, newLocal, local);
if(result == local) break;
local = result;
}
//At this point, local definitely contains a key for lookupValue
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
answered Nov 16 '18 at 13:23
Damien_The_UnbelieverDamien_The_Unbeliever
198k17256347
198k17256347
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53338703%2fmulti-level-lock%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Consider using one of the thread safe collections in
System.Collections.Concurrentinstead.– juharr
Nov 16 '18 at 13:28
What is the relation of # of writes to # of reads ? Are they coarsely the same? Far more Reads than Writes? Far more Writes than Reads? Efficient solutions may depend on that.
– Fildor
Nov 16 '18 at 13:30
More reads than writes, but coarsely the same. I'm using the SharePoint.Client library, iterating through a number of folder names, checking whether each folder exists as a subfolder in SharePoint, and creating it if it doesn't.
– Tam Coton
Nov 16 '18 at 13:34