Split text file into smaller ones by bytes in python
0
I have a problem with splitting a large text file into smaller files by the size (in bytes)
e.g. text file has 30kB, I want to split it into multiple files with 5kB each.
I search a lot but I found almost ways split the file by lines.
any suggestions?
python-3.x text
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
add a comment |
0
I have a problem with splitting a large text file into smaller files by the size (in bytes)
e.g. text file has 30kB, I want to split it into multiple files with 5kB each.
I search a lot but I found almost ways split the file by lines.
any suggestions?
python-3.x text
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
add a comment |
0
0
0
I have a problem with splitting a large text file into smaller files by the size (in bytes)
e.g. text file has 30kB, I want to split it into multiple files with 5kB each.
I search a lot but I found almost ways split the file by lines.
any suggestions?
python-3.x text
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
I have a problem with splitting a large text file into smaller files by the size (in bytes)
e.g. text file has 30kB, I want to split it into multiple files with 5kB each.
I search a lot but I found almost ways split the file by lines.
any suggestions?
python-3.x text
python-3.x text
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
asked Nov 14 '18 at 6:19
Nujud AliNujud Ali
356
356
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
1
If you are looking into splitting it into files of uniform size (e.g. 5KB each), then one solution would be:
- Read the large file as binary
- For every 5000 bytes (5KB), create a new file
- Write those 5000 bytes into the new file
Sample code:
i = 0
with open("large-file", "r", encoding="utf8") as in_file:
bytes = in_file.read(5000) # read 5000 bytes
while bytes:
with open("out-file-" + str(i), 'w', encoding="utf8") as output:
output.write(bytes)
bytes = in_file.read(5000) # read another 5000 bytes
i += 1
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
thanks for your answer, but it doesn't work fine with Unicode texts.
– Nujud Ali
Nov 14 '18 at 7:52
may I ask in what way does it not working? is it the reading or the writing part?
– Andreas
Nov 14 '18 at 7:53
in writing part, when I read segmented files, it views unreadable characters.
– Nujud Ali
Nov 14 '18 at 7:57
can you paste some of the characters here? it might be an issue with the encoding
– Andreas
Nov 14 '18 at 8:03
ط§ظ„ط¥ط¹ظ„ط§ظ† ظ…ط¤ط®ط±ظ‹ط§طŒ ط¹ظ† ظ†ظٹط© ط§ظ„ط¯ط®ظˆظ„ ظپظٹ ط§ط³طھط«ظ…ط§ط± ظ…ط´طھط±ظƒ ظپظٹ ظ…طµظپط§ط© ظƒط¨ظٹط±ط© ط§ظ„طط¬ظ… ط¨ط§ظ„ط³ط§طظ„ ط§ظ„ط؛ط±ط¨ظٹ ظ„ظ„ظ‡ظ†ط¯". ظˆظ‚ط§ظ„ ط§ظ„ظ…ظ‡ظ†ط¯ط³ ط§ظ„ظ†ط§طµط± ط¥ظ† "ط£ط±ط§ظ…ظƒظˆ ط§ظ„ط³ط¹ظˆط¯ظٹط© طھطھظˆط³ط¹ ظپظٹ ط£ط¹ظ…
– Nujud Ali
Nov 14 '18 at 9:38
|
show 1 more comment
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53294188%2fsplit-text-file-into-smaller-ones-by-bytes-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
If you are looking into splitting it into files of uniform size (e.g. 5KB each), then one solution would be:
- Read the large file as binary
- For every 5000 bytes (5KB), create a new file
- Write those 5000 bytes into the new file
Sample code:
i = 0
with open("large-file", "r", encoding="utf8") as in_file:
bytes = in_file.read(5000) # read 5000 bytes
while bytes:
with open("out-file-" + str(i), 'w', encoding="utf8") as output:
output.write(bytes)
bytes = in_file.read(5000) # read another 5000 bytes
i += 1
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
thanks for your answer, but it doesn't work fine with Unicode texts.
– Nujud Ali
Nov 14 '18 at 7:52
may I ask in what way does it not working? is it the reading or the writing part?
– Andreas
Nov 14 '18 at 7:53
in writing part, when I read segmented files, it views unreadable characters.
– Nujud Ali
Nov 14 '18 at 7:57
can you paste some of the characters here? it might be an issue with the encoding
– Andreas
Nov 14 '18 at 8:03
ط§ظ„ط¥ط¹ظ„ط§ظ† ظ…ط¤ط®ط±ظ‹ط§طŒ ط¹ظ† ظ†ظٹط© ط§ظ„ط¯ط®ظˆظ„ ظپظٹ ط§ط³طھط«ظ…ط§ط± ظ…ط´طھط±ظƒ ظپظٹ ظ…طµظپط§ط© ظƒط¨ظٹط±ط© ط§ظ„طط¬ظ… ط¨ط§ظ„ط³ط§طظ„ ط§ظ„ط؛ط±ط¨ظٹ ظ„ظ„ظ‡ظ†ط¯". ظˆظ‚ط§ظ„ ط§ظ„ظ…ظ‡ظ†ط¯ط³ ط§ظ„ظ†ط§طµط± ط¥ظ† "ط£ط±ط§ظ…ظƒظˆ ط§ظ„ط³ط¹ظˆط¯ظٹط© طھطھظˆط³ط¹ ظپظٹ ط£ط¹ظ…
– Nujud Ali
Nov 14 '18 at 9:38
|
show 1 more comment
1
If you are looking into splitting it into files of uniform size (e.g. 5KB each), then one solution would be:
- Read the large file as binary
- For every 5000 bytes (5KB), create a new file
- Write those 5000 bytes into the new file
Sample code:
i = 0
with open("large-file", "r", encoding="utf8") as in_file:
bytes = in_file.read(5000) # read 5000 bytes
while bytes:
with open("out-file-" + str(i), 'w', encoding="utf8") as output:
output.write(bytes)
bytes = in_file.read(5000) # read another 5000 bytes
i += 1
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
thanks for your answer, but it doesn't work fine with Unicode texts.
– Nujud Ali
Nov 14 '18 at 7:52
may I ask in what way does it not working? is it the reading or the writing part?
– Andreas
Nov 14 '18 at 7:53
in writing part, when I read segmented files, it views unreadable characters.
– Nujud Ali
Nov 14 '18 at 7:57
can you paste some of the characters here? it might be an issue with the encoding
– Andreas
Nov 14 '18 at 8:03
ط§ظ„ط¥ط¹ظ„ط§ظ† ظ…ط¤ط®ط±ظ‹ط§طŒ ط¹ظ† ظ†ظٹط© ط§ظ„ط¯ط®ظˆظ„ ظپظٹ ط§ط³طھط«ظ…ط§ط± ظ…ط´طھط±ظƒ ظپظٹ ظ…طµظپط§ط© ظƒط¨ظٹط±ط© ط§ظ„طط¬ظ… ط¨ط§ظ„ط³ط§طظ„ ط§ظ„ط؛ط±ط¨ظٹ ظ„ظ„ظ‡ظ†ط¯". ظˆظ‚ط§ظ„ ط§ظ„ظ…ظ‡ظ†ط¯ط³ ط§ظ„ظ†ط§طµط± ط¥ظ† "ط£ط±ط§ظ…ظƒظˆ ط§ظ„ط³ط¹ظˆط¯ظٹط© طھطھظˆط³ط¹ ظپظٹ ط£ط¹ظ…
– Nujud Ali
Nov 14 '18 at 9:38
|
show 1 more comment
1
1
1
If you are looking into splitting it into files of uniform size (e.g. 5KB each), then one solution would be:
- Read the large file as binary
- For every 5000 bytes (5KB), create a new file
- Write those 5000 bytes into the new file
Sample code:
i = 0
with open("large-file", "r", encoding="utf8") as in_file:
bytes = in_file.read(5000) # read 5000 bytes
while bytes:
with open("out-file-" + str(i), 'w', encoding="utf8") as output:
output.write(bytes)
bytes = in_file.read(5000) # read another 5000 bytes
i += 1
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
If you are looking into splitting it into files of uniform size (e.g. 5KB each), then one solution would be:
- Read the large file as binary
- For every 5000 bytes (5KB), create a new file
- Write those 5000 bytes into the new file
Sample code:
i = 0
with open("large-file", "r", encoding="utf8") as in_file:
bytes = in_file.read(5000) # read 5000 bytes
while bytes:
with open("out-file-" + str(i), 'w', encoding="utf8") as output:
output.write(bytes)
bytes = in_file.read(5000) # read another 5000 bytes
i += 1
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
edited Nov 15 '18 at 1:06
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
answered Nov 14 '18 at 6:34
AndreasAndreas
1,89021018
1,89021018
thanks for your answer, but it doesn't work fine with Unicode texts.
– Nujud Ali
Nov 14 '18 at 7:52
may I ask in what way does it not working? is it the reading or the writing part?
– Andreas
Nov 14 '18 at 7:53
in writing part, when I read segmented files, it views unreadable characters.
– Nujud Ali
Nov 14 '18 at 7:57
can you paste some of the characters here? it might be an issue with the encoding
– Andreas
Nov 14 '18 at 8:03
ط§ظ„ط¥ط¹ظ„ط§ظ† ظ…ط¤ط®ط±ظ‹ط§طŒ ط¹ظ† ظ†ظٹط© ط§ظ„ط¯ط®ظˆظ„ ظپظٹ ط§ط³طھط«ظ…ط§ط± ظ…ط´طھط±ظƒ ظپظٹ ظ…طµظپط§ط© ظƒط¨ظٹط±ط© ط§ظ„طط¬ظ… ط¨ط§ظ„ط³ط§طظ„ ط§ظ„ط؛ط±ط¨ظٹ ظ„ظ„ظ‡ظ†ط¯". ظˆظ‚ط§ظ„ ط§ظ„ظ…ظ‡ظ†ط¯ط³ ط§ظ„ظ†ط§طµط± ط¥ظ† "ط£ط±ط§ظ…ظƒظˆ ط§ظ„ط³ط¹ظˆط¯ظٹط© طھطھظˆط³ط¹ ظپظٹ ط£ط¹ظ…
– Nujud Ali
Nov 14 '18 at 9:38
|
show 1 more comment
thanks for your answer, but it doesn't work fine with Unicode texts.
– Nujud Ali
Nov 14 '18 at 7:52
may I ask in what way does it not working? is it the reading or the writing part?
– Andreas
Nov 14 '18 at 7:53
in writing part, when I read segmented files, it views unreadable characters.
– Nujud Ali
Nov 14 '18 at 7:57
can you paste some of the characters here? it might be an issue with the encoding
– Andreas
Nov 14 '18 at 8:03
ط§ظ„ط¥ط¹ظ„ط§ظ† ظ…ط¤ط®ط±ظ‹ط§طŒ ط¹ظ† ظ†ظٹط© ط§ظ„ط¯ط®ظˆظ„ ظپظٹ ط§ط³طھط«ظ…ط§ط± ظ…ط´طھط±ظƒ ظپظٹ ظ…طµظپط§ط© ظƒط¨ظٹط±ط© ط§ظ„طط¬ظ… ط¨ط§ظ„ط³ط§طظ„ ط§ظ„ط؛ط±ط¨ظٹ ظ„ظ„ظ‡ظ†ط¯". ظˆظ‚ط§ظ„ ط§ظ„ظ…ظ‡ظ†ط¯ط³ ط§ظ„ظ†ط§طµط± ط¥ظ† "ط£ط±ط§ظ…ظƒظˆ ط§ظ„ط³ط¹ظˆط¯ظٹط© طھطھظˆط³ط¹ ظپظٹ ط£ط¹ظ…
– Nujud Ali
Nov 14 '18 at 9:38
thanks for your answer, but it doesn't work fine with Unicode texts.
– Nujud Ali
Nov 14 '18 at 7:52
thanks for your answer, but it doesn't work fine with Unicode texts.
– Nujud Ali
Nov 14 '18 at 7:52
may I ask in what way does it not working? is it the reading or the writing part?
– Andreas
Nov 14 '18 at 7:53
may I ask in what way does it not working? is it the reading or the writing part?
– Andreas
Nov 14 '18 at 7:53
in writing part, when I read segmented files, it views unreadable characters.
– Nujud Ali
Nov 14 '18 at 7:57
in writing part, when I read segmented files, it views unreadable characters.
– Nujud Ali
Nov 14 '18 at 7:57
can you paste some of the characters here? it might be an issue with the encoding
– Andreas
Nov 14 '18 at 8:03
can you paste some of the characters here? it might be an issue with the encoding
– Andreas
Nov 14 '18 at 8:03
ط§ظ„ط¥ط¹ظ„ط§ظ† ظ…ط¤ط®ط±ظ‹ط§طŒ ط¹ظ† ظ†ظٹط© ط§ظ„ط¯ط®ظˆظ„ ظپظٹ ط§ط³طھط«ظ…ط§ط± ظ…ط´طھط±ظƒ ظپظٹ ظ…طµظپط§ط© ظƒط¨ظٹط±ط© ط§ظ„طط¬ظ… ط¨ط§ظ„ط³ط§طظ„ ط§ظ„ط؛ط±ط¨ظٹ ظ„ظ„ظ‡ظ†ط¯". ظˆظ‚ط§ظ„ ط§ظ„ظ…ظ‡ظ†ط¯ط³ ط§ظ„ظ†ط§طµط± ط¥ظ† "ط£ط±ط§ظ…ظƒظˆ ط§ظ„ط³ط¹ظˆط¯ظٹط© طھطھظˆط³ط¹ ظپظٹ ط£ط¹ظ…
– Nujud Ali
Nov 14 '18 at 9:38
ط§ظ„ط¥ط¹ظ„ط§ظ† ظ…ط¤ط®ط±ظ‹ط§طŒ ط¹ظ† ظ†ظٹط© ط§ظ„ط¯ط®ظˆظ„ ظپظٹ ط§ط³طھط«ظ…ط§ط± ظ…ط´طھط±ظƒ ظپظٹ ظ…طµظپط§ط© ظƒط¨ظٹط±ط© ط§ظ„طط¬ظ… ط¨ط§ظ„ط³ط§طظ„ ط§ظ„ط؛ط±ط¨ظٹ ظ„ظ„ظ‡ظ†ط¯". ظˆظ‚ط§ظ„ ط§ظ„ظ…ظ‡ظ†ط¯ط³ ط§ظ„ظ†ط§طµط± ط¥ظ† "ط£ط±ط§ظ…ظƒظˆ ط§ظ„ط³ط¹ظˆط¯ظٹط© طھطھظˆط³ط¹ ظپظٹ ط£ط¹ظ…
– Nujud Ali
Nov 14 '18 at 9:38
|
show 1 more comment
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53294188%2fsplit-text-file-into-smaller-ones-by-bytes-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
