Taking out items from XML C# and performing bulk delete
I have a piece of code which looks like this:
string filePath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + @"DeletedItems" + "DeletedItems" + ".xml";
XDocument xmlDoc = XDocument.Load(filePath);
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
var idsList = FormItemIdList(list);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
This part of the code loads up all items from the XML file:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
Where the XML file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<ItemsToDelete>
<ItemID>113347292264</ItemID>
<ItemID>113334066205</ItemID>
<ItemID>113331816848</ItemID>
<ItemID>113191634415</ItemID>
<ItemID>183480362055</ItemID>
<ItemID>113303425739</ItemID>
<ItemID>112533425202</ItemID>
<ItemID>112007496785</ItemID>
<ItemID>111956371906</ItemID>
<ItemID>112016647700</ItemID>
</ItemsToDelete>
Once the items are loaded into list from C# I form a string which looks like following:
113347292264,113334066205... etc etc
The string is then passed to the stored procedure which looks like following:
create procedure [dbo].[zsp_deleteEndedItems]
(
@ItemIDList nvarchar(max)
)
as
delete from
SearchedUserItems
WHERE EXISTS (SELECT 1 FROM dbo.SplitStringProduction(@ItemIDList,',') S1 WHERE ItemID=S1.val)
Then the SplitStringProduction function is called to break down the passed string and delete the records in DB based on the passed item ID:
create FUNCTION [dbo].[SplitStringProduction]
(
@string nvarchar(max),
@delimiter nvarchar(5)
) RETURNS @t TABLE
(
val nvarchar(500)
)
AS
BEGIN
declare @xml xml
set @xml = N'<root><r>' + replace(@string,@delimiter,'</r><r>') + '</r></root>'
insert into @t(val)
select
r.value('.','varchar(500)') as item
from @xml.nodes('//root/r') as records(r)
RETURN
END
Now, all this works fine, but there are some problems that I'm aware of and that I would like to avoid:
- The list from the XML file can be very long, often containing 100000 + records inside
Now to avoid this I was thinkng to do the following:
- Perform deletion in batches of 5000 records let's say
So the steps would be:
- Pull 5000 items from the XML file
- Pass those 5000 items into the procedure
- Remove the 5000 ItemID's from the XML file
Can someone help me out with this, I'm not sure how to do it ?
c# asp.net linq stored-procedures c#-4.0
add a comment |
I have a piece of code which looks like this:
string filePath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + @"DeletedItems" + "DeletedItems" + ".xml";
XDocument xmlDoc = XDocument.Load(filePath);
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
var idsList = FormItemIdList(list);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
This part of the code loads up all items from the XML file:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
Where the XML file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<ItemsToDelete>
<ItemID>113347292264</ItemID>
<ItemID>113334066205</ItemID>
<ItemID>113331816848</ItemID>
<ItemID>113191634415</ItemID>
<ItemID>183480362055</ItemID>
<ItemID>113303425739</ItemID>
<ItemID>112533425202</ItemID>
<ItemID>112007496785</ItemID>
<ItemID>111956371906</ItemID>
<ItemID>112016647700</ItemID>
</ItemsToDelete>
Once the items are loaded into list from C# I form a string which looks like following:
113347292264,113334066205... etc etc
The string is then passed to the stored procedure which looks like following:
create procedure [dbo].[zsp_deleteEndedItems]
(
@ItemIDList nvarchar(max)
)
as
delete from
SearchedUserItems
WHERE EXISTS (SELECT 1 FROM dbo.SplitStringProduction(@ItemIDList,',') S1 WHERE ItemID=S1.val)
Then the SplitStringProduction function is called to break down the passed string and delete the records in DB based on the passed item ID:
create FUNCTION [dbo].[SplitStringProduction]
(
@string nvarchar(max),
@delimiter nvarchar(5)
) RETURNS @t TABLE
(
val nvarchar(500)
)
AS
BEGIN
declare @xml xml
set @xml = N'<root><r>' + replace(@string,@delimiter,'</r><r>') + '</r></root>'
insert into @t(val)
select
r.value('.','varchar(500)') as item
from @xml.nodes('//root/r') as records(r)
RETURN
END
Now, all this works fine, but there are some problems that I'm aware of and that I would like to avoid:
- The list from the XML file can be very long, often containing 100000 + records inside
Now to avoid this I was thinkng to do the following:
- Perform deletion in batches of 5000 records let's say
So the steps would be:
- Pull 5000 items from the XML file
- Pass those 5000 items into the procedure
- Remove the 5000 ItemID's from the XML file
Can someone help me out with this, I'm not sure how to do it ?
c# asp.net linq stored-procedures c#-4.0
There is a lot of text there, and questions likeBut I'm not sure what is the best way to do it here?
is likely to result in being closed as 'primarily opinion-based'. It might help if you simplify the question a little.
– mjwills
Nov 15 '18 at 11:11
@mjwills I'd simply want to code out those steps at the end that I wrote.. that's pretty much it
– User987
Nov 15 '18 at 11:21
If the first step is not an issue then you can use this to create batches of 5000 of the current list and then loop through the lists and send them in. Else you have to use XmlReader to not load everything into memory and do some more work but the idea would be the same.
– H.Mikhaeljan
Nov 15 '18 at 11:27
add a comment |
I have a piece of code which looks like this:
string filePath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + @"DeletedItems" + "DeletedItems" + ".xml";
XDocument xmlDoc = XDocument.Load(filePath);
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
var idsList = FormItemIdList(list);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
This part of the code loads up all items from the XML file:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
Where the XML file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<ItemsToDelete>
<ItemID>113347292264</ItemID>
<ItemID>113334066205</ItemID>
<ItemID>113331816848</ItemID>
<ItemID>113191634415</ItemID>
<ItemID>183480362055</ItemID>
<ItemID>113303425739</ItemID>
<ItemID>112533425202</ItemID>
<ItemID>112007496785</ItemID>
<ItemID>111956371906</ItemID>
<ItemID>112016647700</ItemID>
</ItemsToDelete>
Once the items are loaded into list from C# I form a string which looks like following:
113347292264,113334066205... etc etc
The string is then passed to the stored procedure which looks like following:
create procedure [dbo].[zsp_deleteEndedItems]
(
@ItemIDList nvarchar(max)
)
as
delete from
SearchedUserItems
WHERE EXISTS (SELECT 1 FROM dbo.SplitStringProduction(@ItemIDList,',') S1 WHERE ItemID=S1.val)
Then the SplitStringProduction function is called to break down the passed string and delete the records in DB based on the passed item ID:
create FUNCTION [dbo].[SplitStringProduction]
(
@string nvarchar(max),
@delimiter nvarchar(5)
) RETURNS @t TABLE
(
val nvarchar(500)
)
AS
BEGIN
declare @xml xml
set @xml = N'<root><r>' + replace(@string,@delimiter,'</r><r>') + '</r></root>'
insert into @t(val)
select
r.value('.','varchar(500)') as item
from @xml.nodes('//root/r') as records(r)
RETURN
END
Now, all this works fine, but there are some problems that I'm aware of and that I would like to avoid:
- The list from the XML file can be very long, often containing 100000 + records inside
Now to avoid this I was thinkng to do the following:
- Perform deletion in batches of 5000 records let's say
So the steps would be:
- Pull 5000 items from the XML file
- Pass those 5000 items into the procedure
- Remove the 5000 ItemID's from the XML file
Can someone help me out with this, I'm not sure how to do it ?
c# asp.net linq stored-procedures c#-4.0
I have a piece of code which looks like this:
string filePath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + @"DeletedItems" + "DeletedItems" + ".xml";
XDocument xmlDoc = XDocument.Load(filePath);
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
var idsList = FormItemIdList(list);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
This part of the code loads up all items from the XML file:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
Where the XML file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<ItemsToDelete>
<ItemID>113347292264</ItemID>
<ItemID>113334066205</ItemID>
<ItemID>113331816848</ItemID>
<ItemID>113191634415</ItemID>
<ItemID>183480362055</ItemID>
<ItemID>113303425739</ItemID>
<ItemID>112533425202</ItemID>
<ItemID>112007496785</ItemID>
<ItemID>111956371906</ItemID>
<ItemID>112016647700</ItemID>
</ItemsToDelete>
Once the items are loaded into list from C# I form a string which looks like following:
113347292264,113334066205... etc etc
The string is then passed to the stored procedure which looks like following:
create procedure [dbo].[zsp_deleteEndedItems]
(
@ItemIDList nvarchar(max)
)
as
delete from
SearchedUserItems
WHERE EXISTS (SELECT 1 FROM dbo.SplitStringProduction(@ItemIDList,',') S1 WHERE ItemID=S1.val)
Then the SplitStringProduction function is called to break down the passed string and delete the records in DB based on the passed item ID:
create FUNCTION [dbo].[SplitStringProduction]
(
@string nvarchar(max),
@delimiter nvarchar(5)
) RETURNS @t TABLE
(
val nvarchar(500)
)
AS
BEGIN
declare @xml xml
set @xml = N'<root><r>' + replace(@string,@delimiter,'</r><r>') + '</r></root>'
insert into @t(val)
select
r.value('.','varchar(500)') as item
from @xml.nodes('//root/r') as records(r)
RETURN
END
Now, all this works fine, but there are some problems that I'm aware of and that I would like to avoid:
- The list from the XML file can be very long, often containing 100000 + records inside
Now to avoid this I was thinkng to do the following:
- Perform deletion in batches of 5000 records let's say
So the steps would be:
- Pull 5000 items from the XML file
- Pass those 5000 items into the procedure
- Remove the 5000 ItemID's from the XML file
Can someone help me out with this, I'm not sure how to do it ?
c# asp.net linq stored-procedures c#-4.0
c# asp.net linq stored-procedures c#-4.0
edited Nov 15 '18 at 11:20
User987
asked Nov 15 '18 at 10:57
User987User987
1,35932147
1,35932147
There is a lot of text there, and questions likeBut I'm not sure what is the best way to do it here?
is likely to result in being closed as 'primarily opinion-based'. It might help if you simplify the question a little.
– mjwills
Nov 15 '18 at 11:11
@mjwills I'd simply want to code out those steps at the end that I wrote.. that's pretty much it
– User987
Nov 15 '18 at 11:21
If the first step is not an issue then you can use this to create batches of 5000 of the current list and then loop through the lists and send them in. Else you have to use XmlReader to not load everything into memory and do some more work but the idea would be the same.
– H.Mikhaeljan
Nov 15 '18 at 11:27
add a comment |
There is a lot of text there, and questions likeBut I'm not sure what is the best way to do it here?
is likely to result in being closed as 'primarily opinion-based'. It might help if you simplify the question a little.
– mjwills
Nov 15 '18 at 11:11
@mjwills I'd simply want to code out those steps at the end that I wrote.. that's pretty much it
– User987
Nov 15 '18 at 11:21
If the first step is not an issue then you can use this to create batches of 5000 of the current list and then loop through the lists and send them in. Else you have to use XmlReader to not load everything into memory and do some more work but the idea would be the same.
– H.Mikhaeljan
Nov 15 '18 at 11:27
There is a lot of text there, and questions like
But I'm not sure what is the best way to do it here?
is likely to result in being closed as 'primarily opinion-based'. It might help if you simplify the question a little.– mjwills
Nov 15 '18 at 11:11
There is a lot of text there, and questions like
But I'm not sure what is the best way to do it here?
is likely to result in being closed as 'primarily opinion-based'. It might help if you simplify the question a little.– mjwills
Nov 15 '18 at 11:11
@mjwills I'd simply want to code out those steps at the end that I wrote.. that's pretty much it
– User987
Nov 15 '18 at 11:21
@mjwills I'd simply want to code out those steps at the end that I wrote.. that's pretty much it
– User987
Nov 15 '18 at 11:21
If the first step is not an issue then you can use this to create batches of 5000 of the current list and then loop through the lists and send them in. Else you have to use XmlReader to not load everything into memory and do some more work but the idea would be the same.
– H.Mikhaeljan
Nov 15 '18 at 11:27
If the first step is not an issue then you can use this to create batches of 5000 of the current list and then loop through the lists and send them in. Else you have to use XmlReader to not load everything into memory and do some more work but the idea would be the same.
– H.Mikhaeljan
Nov 15 '18 at 11:27
add a comment |
1 Answer
1
active
oldest
votes
It sounds like you just want to split the list into smaller chunks:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
while(list.Any())
{
var subList = list.Take(5000);
var idsList = FormItemIdList(subList);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
list.RemoveRange(subList);
}
Haha this is exactly it :)
– User987
Nov 15 '18 at 11:29
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53317915%2ftaking-out-items-from-xml-c-sharp-and-performing-bulk-delete%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
It sounds like you just want to split the list into smaller chunks:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
while(list.Any())
{
var subList = list.Take(5000);
var idsList = FormItemIdList(subList);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
list.RemoveRange(subList);
}
Haha this is exactly it :)
– User987
Nov 15 '18 at 11:29
add a comment |
It sounds like you just want to split the list into smaller chunks:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
while(list.Any())
{
var subList = list.Take(5000);
var idsList = FormItemIdList(subList);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
list.RemoveRange(subList);
}
Haha this is exactly it :)
– User987
Nov 15 '18 at 11:29
add a comment |
It sounds like you just want to split the list into smaller chunks:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
while(list.Any())
{
var subList = list.Take(5000);
var idsList = FormItemIdList(subList);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
list.RemoveRange(subList);
}
It sounds like you just want to split the list into smaller chunks:
var list = xmlDoc.Root.Elements("ItemID")
.Select(element => element.Value)
.ToList();
while(list.Any())
{
var subList = list.Take(5000);
var idsList = FormItemIdList(subList);
ctx.zsp_deleteEndedItems(idsList);
ctx.zsp_deleteEndedItemsTransactions(idsList);
list.RemoveRange(subList);
}
answered Nov 15 '18 at 11:28
NeilNeil
4,85711537
4,85711537
Haha this is exactly it :)
– User987
Nov 15 '18 at 11:29
add a comment |
Haha this is exactly it :)
– User987
Nov 15 '18 at 11:29
Haha this is exactly it :)
– User987
Nov 15 '18 at 11:29
Haha this is exactly it :)
– User987
Nov 15 '18 at 11:29
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53317915%2ftaking-out-items-from-xml-c-sharp-and-performing-bulk-delete%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
There is a lot of text there, and questions like
But I'm not sure what is the best way to do it here?
is likely to result in being closed as 'primarily opinion-based'. It might help if you simplify the question a little.– mjwills
Nov 15 '18 at 11:11
@mjwills I'd simply want to code out those steps at the end that I wrote.. that's pretty much it
– User987
Nov 15 '18 at 11:21
If the first step is not an issue then you can use this to create batches of 5000 of the current list and then loop through the lists and send them in. Else you have to use XmlReader to not load everything into memory and do some more work but the idea would be the same.
– H.Mikhaeljan
Nov 15 '18 at 11:27