Issue with headers while splitting CSV file [Python 3]
I am newbie here in StackOverflow, so if made any formal mistakes with that post, correct me pls, would be appreciated!
However, coming back to main topic: I have some issues with headers while splitting big CSV file into smaller ones. General idea is to split mentioned file according to the 1 column and create smaller files with column names, for instance:
Fruit Country Color
apple Poland red
banana Argentina yellow
pineapple Argentina brown
pear Poland green
melon Turkey yellow
plum Poland violet
peach Turkey orange
grenade Argentina violet
Code should generate 3 different files (Poland.csv, Turkey.csv, Argentina.csv)
So far I've made the below code which is splitting CSV correctly but cannot append headers properly (they are added through each iteration). Do you have any ideas how can I deal with it?
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(header)
writer.writerow(row)
myfile.close()
splitter(csvreader)
python csv split header
add a comment |
I am newbie here in StackOverflow, so if made any formal mistakes with that post, correct me pls, would be appreciated!
However, coming back to main topic: I have some issues with headers while splitting big CSV file into smaller ones. General idea is to split mentioned file according to the 1 column and create smaller files with column names, for instance:
Fruit Country Color
apple Poland red
banana Argentina yellow
pineapple Argentina brown
pear Poland green
melon Turkey yellow
plum Poland violet
peach Turkey orange
grenade Argentina violet
Code should generate 3 different files (Poland.csv, Turkey.csv, Argentina.csv)
So far I've made the below code which is splitting CSV correctly but cannot append headers properly (they are added through each iteration). Do you have any ideas how can I deal with it?
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(header)
writer.writerow(row)
myfile.close()
splitter(csvreader)
python csv split header
1
Of course they're added through each iteration the line to add it is in the loop. If you're going row by row then I suppose you need to check whether or not that file has already been opened in your script. If it has, don't write the header again. If it hasn't, write the header.
– Neil
Nov 12 at 12:46
add a comment |
I am newbie here in StackOverflow, so if made any formal mistakes with that post, correct me pls, would be appreciated!
However, coming back to main topic: I have some issues with headers while splitting big CSV file into smaller ones. General idea is to split mentioned file according to the 1 column and create smaller files with column names, for instance:
Fruit Country Color
apple Poland red
banana Argentina yellow
pineapple Argentina brown
pear Poland green
melon Turkey yellow
plum Poland violet
peach Turkey orange
grenade Argentina violet
Code should generate 3 different files (Poland.csv, Turkey.csv, Argentina.csv)
So far I've made the below code which is splitting CSV correctly but cannot append headers properly (they are added through each iteration). Do you have any ideas how can I deal with it?
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(header)
writer.writerow(row)
myfile.close()
splitter(csvreader)
python csv split header
I am newbie here in StackOverflow, so if made any formal mistakes with that post, correct me pls, would be appreciated!
However, coming back to main topic: I have some issues with headers while splitting big CSV file into smaller ones. General idea is to split mentioned file according to the 1 column and create smaller files with column names, for instance:
Fruit Country Color
apple Poland red
banana Argentina yellow
pineapple Argentina brown
pear Poland green
melon Turkey yellow
plum Poland violet
peach Turkey orange
grenade Argentina violet
Code should generate 3 different files (Poland.csv, Turkey.csv, Argentina.csv)
So far I've made the below code which is splitting CSV correctly but cannot append headers properly (they are added through each iteration). Do you have any ideas how can I deal with it?
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(header)
writer.writerow(row)
myfile.close()
splitter(csvreader)
python csv split header
python csv split header
edited Nov 12 at 12:52
has
746517
746517
asked Nov 12 at 12:29
ArPy
31
31
1
Of course they're added through each iteration the line to add it is in the loop. If you're going row by row then I suppose you need to check whether or not that file has already been opened in your script. If it has, don't write the header again. If it hasn't, write the header.
– Neil
Nov 12 at 12:46
add a comment |
1
Of course they're added through each iteration the line to add it is in the loop. If you're going row by row then I suppose you need to check whether or not that file has already been opened in your script. If it has, don't write the header again. If it hasn't, write the header.
– Neil
Nov 12 at 12:46
1
1
Of course they're added through each iteration the line to add it is in the loop. If you're going row by row then I suppose you need to check whether or not that file has already been opened in your script. If it has, don't write the header again. If it hasn't, write the header.
– Neil
Nov 12 at 12:46
Of course they're added through each iteration the line to add it is in the loop. If you're going row by row then I suppose you need to check whether or not that file has already been opened in your script. If it has, don't write the header again. If it hasn't, write the header.
– Neil
Nov 12 at 12:46
add a comment |
2 Answers
2
active
oldest
votes
Try something like this (quick and dirty but should work):
def splitter(u):
filenames_already_opened = # Just keep a list of the csv's you've already created and therefore have added a header to.
for row in u:
filename = row[1] + '.csv'
with open(filename, 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if filename in filenames_already_opened: # Don't add a header if it's already got one.
pass
else:
writer.writerow(header)
filenames_already_opened.append(filename)
writer.writerow(row)
myfile.close()
This works as well, thank you ! :)
– ArPy
Nov 12 at 19:28
add a comment |
This fixes the problem:
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
tableNames =
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if not row[1] in tableNames:
writer.writerow(header)
tableNames.append(row[1])
writer.writerow(row)
myfile.close()
splitter(csvreader)
Thank you very much! Indeed, it solved my issue :)
– ArPy
Nov 12 at 15:53
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262230%2fissue-with-headers-while-splitting-csv-file-python-3%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Try something like this (quick and dirty but should work):
def splitter(u):
filenames_already_opened = # Just keep a list of the csv's you've already created and therefore have added a header to.
for row in u:
filename = row[1] + '.csv'
with open(filename, 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if filename in filenames_already_opened: # Don't add a header if it's already got one.
pass
else:
writer.writerow(header)
filenames_already_opened.append(filename)
writer.writerow(row)
myfile.close()
This works as well, thank you ! :)
– ArPy
Nov 12 at 19:28
add a comment |
Try something like this (quick and dirty but should work):
def splitter(u):
filenames_already_opened = # Just keep a list of the csv's you've already created and therefore have added a header to.
for row in u:
filename = row[1] + '.csv'
with open(filename, 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if filename in filenames_already_opened: # Don't add a header if it's already got one.
pass
else:
writer.writerow(header)
filenames_already_opened.append(filename)
writer.writerow(row)
myfile.close()
This works as well, thank you ! :)
– ArPy
Nov 12 at 19:28
add a comment |
Try something like this (quick and dirty but should work):
def splitter(u):
filenames_already_opened = # Just keep a list of the csv's you've already created and therefore have added a header to.
for row in u:
filename = row[1] + '.csv'
with open(filename, 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if filename in filenames_already_opened: # Don't add a header if it's already got one.
pass
else:
writer.writerow(header)
filenames_already_opened.append(filename)
writer.writerow(row)
myfile.close()
Try something like this (quick and dirty but should work):
def splitter(u):
filenames_already_opened = # Just keep a list of the csv's you've already created and therefore have added a header to.
for row in u:
filename = row[1] + '.csv'
with open(filename, 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if filename in filenames_already_opened: # Don't add a header if it's already got one.
pass
else:
writer.writerow(header)
filenames_already_opened.append(filename)
writer.writerow(row)
myfile.close()
answered Nov 12 at 12:49
Neil
51119
51119
This works as well, thank you ! :)
– ArPy
Nov 12 at 19:28
add a comment |
This works as well, thank you ! :)
– ArPy
Nov 12 at 19:28
This works as well, thank you ! :)
– ArPy
Nov 12 at 19:28
This works as well, thank you ! :)
– ArPy
Nov 12 at 19:28
add a comment |
This fixes the problem:
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
tableNames =
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if not row[1] in tableNames:
writer.writerow(header)
tableNames.append(row[1])
writer.writerow(row)
myfile.close()
splitter(csvreader)
Thank you very much! Indeed, it solved my issue :)
– ArPy
Nov 12 at 15:53
add a comment |
This fixes the problem:
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
tableNames =
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if not row[1] in tableNames:
writer.writerow(header)
tableNames.append(row[1])
writer.writerow(row)
myfile.close()
splitter(csvreader)
Thank you very much! Indeed, it solved my issue :)
– ArPy
Nov 12 at 15:53
add a comment |
This fixes the problem:
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
tableNames =
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if not row[1] in tableNames:
writer.writerow(header)
tableNames.append(row[1])
writer.writerow(row)
myfile.close()
splitter(csvreader)
This fixes the problem:
import csv
opener = open('file.csv', 'r', encoding='utf-8')
csvreader = csv.reader(opener, delimiter=';')
header = next(csvreader)
def splitter(u):
tableNames =
for row in u:
with open(row[1] + '.csv', 'a', encoding='utf-8', newline='') as myfile:
writer = csv.writer(myfile, delimiter=';', quotechar='|', quoting=csv.QUOTE_MINIMAL)
if not row[1] in tableNames:
writer.writerow(header)
tableNames.append(row[1])
writer.writerow(row)
myfile.close()
splitter(csvreader)
answered Nov 12 at 12:51
Mr_Z
16616
16616
Thank you very much! Indeed, it solved my issue :)
– ArPy
Nov 12 at 15:53
add a comment |
Thank you very much! Indeed, it solved my issue :)
– ArPy
Nov 12 at 15:53
Thank you very much! Indeed, it solved my issue :)
– ArPy
Nov 12 at 15:53
Thank you very much! Indeed, it solved my issue :)
– ArPy
Nov 12 at 15:53
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262230%2fissue-with-headers-while-splitting-csv-file-python-3%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Of course they're added through each iteration the line to add it is in the loop. If you're going row by row then I suppose you need to check whether or not that file has already been opened in your script. If it has, don't write the header again. If it hasn't, write the header.
– Neil
Nov 12 at 12:46