Get a subset of a data frame into a matrix
I have this data frame:
I want just the numbers under August - September to be placed into a matrix, how can I do this?
I tried this cf = df.iloc[:,1:12]
which gives me but it gives me the headers as well which I do not need.
I did this
cf = df.iloc[:,1:12]
cf = cf.values
print(cf)
which gives me
[['$0.00 ' '$771.98 ' '$0.00 ' ..., '$771.98 ' '$0.00 ' '$1,543.96 ']
['$1,320.83 ' '$4,782.33 ' '$1,320.83 ' ..., '$1,954.45 ' '$0.00 '
'$1,954.45 ']
['$2,043.61 ' '$0.00 ' '$4,087.22 ' ..., '$4,662.30 ' '$2,907.82 '
'$1,549.53 ']
...,
['$427.60 ' '$0.00 ' '$427.60 ' ..., '$427.60 ' '$0.00 ' '$427.60 ']
['$868.58 ' '$1,737.16 ' '$0.00 ' ..., '$868.58 ' '$868.58 ' '$868.58 ']
['$0.00 ' '$1,590.07 ' '$0.00 ' ..., '$787.75 ' '$0.00 ' '$0.00 ']]
I need these to be of floating types.
python pandas
add a comment |
I have this data frame:
I want just the numbers under August - September to be placed into a matrix, how can I do this?
I tried this cf = df.iloc[:,1:12]
which gives me but it gives me the headers as well which I do not need.
I did this
cf = df.iloc[:,1:12]
cf = cf.values
print(cf)
which gives me
[['$0.00 ' '$771.98 ' '$0.00 ' ..., '$771.98 ' '$0.00 ' '$1,543.96 ']
['$1,320.83 ' '$4,782.33 ' '$1,320.83 ' ..., '$1,954.45 ' '$0.00 '
'$1,954.45 ']
['$2,043.61 ' '$0.00 ' '$4,087.22 ' ..., '$4,662.30 ' '$2,907.82 '
'$1,549.53 ']
...,
['$427.60 ' '$0.00 ' '$427.60 ' ..., '$427.60 ' '$0.00 ' '$427.60 ']
['$868.58 ' '$1,737.16 ' '$0.00 ' ..., '$868.58 ' '$868.58 ' '$868.58 ']
['$0.00 ' '$1,590.07 ' '$0.00 ' ..., '$787.75 ' '$0.00 ' '$0.00 ']]
I need these to be of floating types.
python pandas
Depending on your pandas version, pandas.pydata.org/pandas-docs/stable/generated/… or pandas.pydata.org/pandas-docs/stable/generated/… should do the trick. (You'll need to subset first, of course.)
– Evan
Nov 15 '18 at 23:17
@Evan Those did not really help me but thanks
– Snorrlaxxx
Nov 15 '18 at 23:22
add a comment |
I have this data frame:
I want just the numbers under August - September to be placed into a matrix, how can I do this?
I tried this cf = df.iloc[:,1:12]
which gives me but it gives me the headers as well which I do not need.
I did this
cf = df.iloc[:,1:12]
cf = cf.values
print(cf)
which gives me
[['$0.00 ' '$771.98 ' '$0.00 ' ..., '$771.98 ' '$0.00 ' '$1,543.96 ']
['$1,320.83 ' '$4,782.33 ' '$1,320.83 ' ..., '$1,954.45 ' '$0.00 '
'$1,954.45 ']
['$2,043.61 ' '$0.00 ' '$4,087.22 ' ..., '$4,662.30 ' '$2,907.82 '
'$1,549.53 ']
...,
['$427.60 ' '$0.00 ' '$427.60 ' ..., '$427.60 ' '$0.00 ' '$427.60 ']
['$868.58 ' '$1,737.16 ' '$0.00 ' ..., '$868.58 ' '$868.58 ' '$868.58 ']
['$0.00 ' '$1,590.07 ' '$0.00 ' ..., '$787.75 ' '$0.00 ' '$0.00 ']]
I need these to be of floating types.
python pandas
I have this data frame:
I want just the numbers under August - September to be placed into a matrix, how can I do this?
I tried this cf = df.iloc[:,1:12]
which gives me but it gives me the headers as well which I do not need.
I did this
cf = df.iloc[:,1:12]
cf = cf.values
print(cf)
which gives me
[['$0.00 ' '$771.98 ' '$0.00 ' ..., '$771.98 ' '$0.00 ' '$1,543.96 ']
['$1,320.83 ' '$4,782.33 ' '$1,320.83 ' ..., '$1,954.45 ' '$0.00 '
'$1,954.45 ']
['$2,043.61 ' '$0.00 ' '$4,087.22 ' ..., '$4,662.30 ' '$2,907.82 '
'$1,549.53 ']
...,
['$427.60 ' '$0.00 ' '$427.60 ' ..., '$427.60 ' '$0.00 ' '$427.60 ']
['$868.58 ' '$1,737.16 ' '$0.00 ' ..., '$868.58 ' '$868.58 ' '$868.58 ']
['$0.00 ' '$1,590.07 ' '$0.00 ' ..., '$787.75 ' '$0.00 ' '$0.00 ']]
I need these to be of floating types.
python pandas
python pandas
edited Nov 15 '18 at 23:52
Snorrlaxxx
asked Nov 15 '18 at 23:13
SnorrlaxxxSnorrlaxxx
13611
13611
Depending on your pandas version, pandas.pydata.org/pandas-docs/stable/generated/… or pandas.pydata.org/pandas-docs/stable/generated/… should do the trick. (You'll need to subset first, of course.)
– Evan
Nov 15 '18 at 23:17
@Evan Those did not really help me but thanks
– Snorrlaxxx
Nov 15 '18 at 23:22
add a comment |
Depending on your pandas version, pandas.pydata.org/pandas-docs/stable/generated/… or pandas.pydata.org/pandas-docs/stable/generated/… should do the trick. (You'll need to subset first, of course.)
– Evan
Nov 15 '18 at 23:17
@Evan Those did not really help me but thanks
– Snorrlaxxx
Nov 15 '18 at 23:22
Depending on your pandas version, pandas.pydata.org/pandas-docs/stable/generated/… or pandas.pydata.org/pandas-docs/stable/generated/… should do the trick. (You'll need to subset first, of course.)
– Evan
Nov 15 '18 at 23:17
Depending on your pandas version, pandas.pydata.org/pandas-docs/stable/generated/… or pandas.pydata.org/pandas-docs/stable/generated/… should do the trick. (You'll need to subset first, of course.)
– Evan
Nov 15 '18 at 23:17
@Evan Those did not really help me but thanks
– Snorrlaxxx
Nov 15 '18 at 23:22
@Evan Those did not really help me but thanks
– Snorrlaxxx
Nov 15 '18 at 23:22
add a comment |
1 Answer
1
active
oldest
votes
Given (stealing from an earlier problem today):
"""
IndexID IndexDateTime IndexAttribute ColumnA ColumnB
1 2015-02-05 8 A B
1 2015-02-05 7 C D
1 2015-02-10 7 X Y
"""
import pandas as pd
import numpy as np
df = pd.read_clipboard(parse_dates=["IndexDateTime"]).set_index(["IndexID", "IndexDateTime", "IndexAttribute"])
df:
ColumnA ColumnB
IndexID IndexDateTime IndexAttribute
1 2015-02-05 8 A B
7 C D
2015-02-10 7 X Y
using
df.values
returns
array([['A', 'B'],
['C', 'D'],
['X', 'Y']], dtype=object)
To subset, you can use a couple of techniques. Here,
df.loc[:, "ColumnA":"ColumnB"]
Returns all rows and slices from ColumnA
to ColumnB
. Other options include syntax like df[df["column"] == condition]
or df.iloc[1:3, 0:5]
; which approach is best more or less depends on your data, how readable you want your code to be, and which is fastest for what you're trying to do. Using .loc
or .iloc
is usually a safe bet, in my experience.
In general for pandas problems, it is helpful to post some sample data rather than an image of your dataframe; otherwise, the burden is on the SO users to generate data that mimics yours.
Edit:
To convert currency to float, try this:
df.replace('[$,]', '', regex=True).astype(float)
So, in a one-liner,
df.loc[:, "ColumnB":"ColumnC"].replace('[$,]', '', regex=True).values.astype(float)
yields
array([[1.23, 1.23],
[1.23, 1.23],
[1.23, 1.23]])
Can you see my edit?
– Snorrlaxxx
Nov 15 '18 at 23:53
Edited. Portions of your question were already answered elsewhere on SO; as such, your question isn't an exact duplicate, but I think you should spend some time googling SO to see what other people have tried to address smaller segments of the project you're working on.
– Evan
Nov 16 '18 at 0:10
ValueError: could not convert string to float: '(641.99)'
– Snorrlaxxx
Nov 16 '18 at 0:16
Ah, the()
tripped up the replacement values. If you're dealing with accounting data, I think that might be a negative number... so you'll need to adjust accordingly. See: stackoverflow.com/questions/31521526/…
– Evan
Nov 16 '18 at 0:24
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53329187%2fget-a-subset-of-a-data-frame-into-a-matrix%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Given (stealing from an earlier problem today):
"""
IndexID IndexDateTime IndexAttribute ColumnA ColumnB
1 2015-02-05 8 A B
1 2015-02-05 7 C D
1 2015-02-10 7 X Y
"""
import pandas as pd
import numpy as np
df = pd.read_clipboard(parse_dates=["IndexDateTime"]).set_index(["IndexID", "IndexDateTime", "IndexAttribute"])
df:
ColumnA ColumnB
IndexID IndexDateTime IndexAttribute
1 2015-02-05 8 A B
7 C D
2015-02-10 7 X Y
using
df.values
returns
array([['A', 'B'],
['C', 'D'],
['X', 'Y']], dtype=object)
To subset, you can use a couple of techniques. Here,
df.loc[:, "ColumnA":"ColumnB"]
Returns all rows and slices from ColumnA
to ColumnB
. Other options include syntax like df[df["column"] == condition]
or df.iloc[1:3, 0:5]
; which approach is best more or less depends on your data, how readable you want your code to be, and which is fastest for what you're trying to do. Using .loc
or .iloc
is usually a safe bet, in my experience.
In general for pandas problems, it is helpful to post some sample data rather than an image of your dataframe; otherwise, the burden is on the SO users to generate data that mimics yours.
Edit:
To convert currency to float, try this:
df.replace('[$,]', '', regex=True).astype(float)
So, in a one-liner,
df.loc[:, "ColumnB":"ColumnC"].replace('[$,]', '', regex=True).values.astype(float)
yields
array([[1.23, 1.23],
[1.23, 1.23],
[1.23, 1.23]])
Can you see my edit?
– Snorrlaxxx
Nov 15 '18 at 23:53
Edited. Portions of your question were already answered elsewhere on SO; as such, your question isn't an exact duplicate, but I think you should spend some time googling SO to see what other people have tried to address smaller segments of the project you're working on.
– Evan
Nov 16 '18 at 0:10
ValueError: could not convert string to float: '(641.99)'
– Snorrlaxxx
Nov 16 '18 at 0:16
Ah, the()
tripped up the replacement values. If you're dealing with accounting data, I think that might be a negative number... so you'll need to adjust accordingly. See: stackoverflow.com/questions/31521526/…
– Evan
Nov 16 '18 at 0:24
add a comment |
Given (stealing from an earlier problem today):
"""
IndexID IndexDateTime IndexAttribute ColumnA ColumnB
1 2015-02-05 8 A B
1 2015-02-05 7 C D
1 2015-02-10 7 X Y
"""
import pandas as pd
import numpy as np
df = pd.read_clipboard(parse_dates=["IndexDateTime"]).set_index(["IndexID", "IndexDateTime", "IndexAttribute"])
df:
ColumnA ColumnB
IndexID IndexDateTime IndexAttribute
1 2015-02-05 8 A B
7 C D
2015-02-10 7 X Y
using
df.values
returns
array([['A', 'B'],
['C', 'D'],
['X', 'Y']], dtype=object)
To subset, you can use a couple of techniques. Here,
df.loc[:, "ColumnA":"ColumnB"]
Returns all rows and slices from ColumnA
to ColumnB
. Other options include syntax like df[df["column"] == condition]
or df.iloc[1:3, 0:5]
; which approach is best more or less depends on your data, how readable you want your code to be, and which is fastest for what you're trying to do. Using .loc
or .iloc
is usually a safe bet, in my experience.
In general for pandas problems, it is helpful to post some sample data rather than an image of your dataframe; otherwise, the burden is on the SO users to generate data that mimics yours.
Edit:
To convert currency to float, try this:
df.replace('[$,]', '', regex=True).astype(float)
So, in a one-liner,
df.loc[:, "ColumnB":"ColumnC"].replace('[$,]', '', regex=True).values.astype(float)
yields
array([[1.23, 1.23],
[1.23, 1.23],
[1.23, 1.23]])
Can you see my edit?
– Snorrlaxxx
Nov 15 '18 at 23:53
Edited. Portions of your question were already answered elsewhere on SO; as such, your question isn't an exact duplicate, but I think you should spend some time googling SO to see what other people have tried to address smaller segments of the project you're working on.
– Evan
Nov 16 '18 at 0:10
ValueError: could not convert string to float: '(641.99)'
– Snorrlaxxx
Nov 16 '18 at 0:16
Ah, the()
tripped up the replacement values. If you're dealing with accounting data, I think that might be a negative number... so you'll need to adjust accordingly. See: stackoverflow.com/questions/31521526/…
– Evan
Nov 16 '18 at 0:24
add a comment |
Given (stealing from an earlier problem today):
"""
IndexID IndexDateTime IndexAttribute ColumnA ColumnB
1 2015-02-05 8 A B
1 2015-02-05 7 C D
1 2015-02-10 7 X Y
"""
import pandas as pd
import numpy as np
df = pd.read_clipboard(parse_dates=["IndexDateTime"]).set_index(["IndexID", "IndexDateTime", "IndexAttribute"])
df:
ColumnA ColumnB
IndexID IndexDateTime IndexAttribute
1 2015-02-05 8 A B
7 C D
2015-02-10 7 X Y
using
df.values
returns
array([['A', 'B'],
['C', 'D'],
['X', 'Y']], dtype=object)
To subset, you can use a couple of techniques. Here,
df.loc[:, "ColumnA":"ColumnB"]
Returns all rows and slices from ColumnA
to ColumnB
. Other options include syntax like df[df["column"] == condition]
or df.iloc[1:3, 0:5]
; which approach is best more or less depends on your data, how readable you want your code to be, and which is fastest for what you're trying to do. Using .loc
or .iloc
is usually a safe bet, in my experience.
In general for pandas problems, it is helpful to post some sample data rather than an image of your dataframe; otherwise, the burden is on the SO users to generate data that mimics yours.
Edit:
To convert currency to float, try this:
df.replace('[$,]', '', regex=True).astype(float)
So, in a one-liner,
df.loc[:, "ColumnB":"ColumnC"].replace('[$,]', '', regex=True).values.astype(float)
yields
array([[1.23, 1.23],
[1.23, 1.23],
[1.23, 1.23]])
Given (stealing from an earlier problem today):
"""
IndexID IndexDateTime IndexAttribute ColumnA ColumnB
1 2015-02-05 8 A B
1 2015-02-05 7 C D
1 2015-02-10 7 X Y
"""
import pandas as pd
import numpy as np
df = pd.read_clipboard(parse_dates=["IndexDateTime"]).set_index(["IndexID", "IndexDateTime", "IndexAttribute"])
df:
ColumnA ColumnB
IndexID IndexDateTime IndexAttribute
1 2015-02-05 8 A B
7 C D
2015-02-10 7 X Y
using
df.values
returns
array([['A', 'B'],
['C', 'D'],
['X', 'Y']], dtype=object)
To subset, you can use a couple of techniques. Here,
df.loc[:, "ColumnA":"ColumnB"]
Returns all rows and slices from ColumnA
to ColumnB
. Other options include syntax like df[df["column"] == condition]
or df.iloc[1:3, 0:5]
; which approach is best more or less depends on your data, how readable you want your code to be, and which is fastest for what you're trying to do. Using .loc
or .iloc
is usually a safe bet, in my experience.
In general for pandas problems, it is helpful to post some sample data rather than an image of your dataframe; otherwise, the burden is on the SO users to generate data that mimics yours.
Edit:
To convert currency to float, try this:
df.replace('[$,]', '', regex=True).astype(float)
So, in a one-liner,
df.loc[:, "ColumnB":"ColumnC"].replace('[$,]', '', regex=True).values.astype(float)
yields
array([[1.23, 1.23],
[1.23, 1.23],
[1.23, 1.23]])
edited Nov 16 '18 at 0:08
answered Nov 15 '18 at 23:27
EvanEvan
1,141516
1,141516
Can you see my edit?
– Snorrlaxxx
Nov 15 '18 at 23:53
Edited. Portions of your question were already answered elsewhere on SO; as such, your question isn't an exact duplicate, but I think you should spend some time googling SO to see what other people have tried to address smaller segments of the project you're working on.
– Evan
Nov 16 '18 at 0:10
ValueError: could not convert string to float: '(641.99)'
– Snorrlaxxx
Nov 16 '18 at 0:16
Ah, the()
tripped up the replacement values. If you're dealing with accounting data, I think that might be a negative number... so you'll need to adjust accordingly. See: stackoverflow.com/questions/31521526/…
– Evan
Nov 16 '18 at 0:24
add a comment |
Can you see my edit?
– Snorrlaxxx
Nov 15 '18 at 23:53
Edited. Portions of your question were already answered elsewhere on SO; as such, your question isn't an exact duplicate, but I think you should spend some time googling SO to see what other people have tried to address smaller segments of the project you're working on.
– Evan
Nov 16 '18 at 0:10
ValueError: could not convert string to float: '(641.99)'
– Snorrlaxxx
Nov 16 '18 at 0:16
Ah, the()
tripped up the replacement values. If you're dealing with accounting data, I think that might be a negative number... so you'll need to adjust accordingly. See: stackoverflow.com/questions/31521526/…
– Evan
Nov 16 '18 at 0:24
Can you see my edit?
– Snorrlaxxx
Nov 15 '18 at 23:53
Can you see my edit?
– Snorrlaxxx
Nov 15 '18 at 23:53
Edited. Portions of your question were already answered elsewhere on SO; as such, your question isn't an exact duplicate, but I think you should spend some time googling SO to see what other people have tried to address smaller segments of the project you're working on.
– Evan
Nov 16 '18 at 0:10
Edited. Portions of your question were already answered elsewhere on SO; as such, your question isn't an exact duplicate, but I think you should spend some time googling SO to see what other people have tried to address smaller segments of the project you're working on.
– Evan
Nov 16 '18 at 0:10
ValueError: could not convert string to float: '(641.99)'
– Snorrlaxxx
Nov 16 '18 at 0:16
ValueError: could not convert string to float: '(641.99)'
– Snorrlaxxx
Nov 16 '18 at 0:16
Ah, the
()
tripped up the replacement values. If you're dealing with accounting data, I think that might be a negative number... so you'll need to adjust accordingly. See: stackoverflow.com/questions/31521526/…– Evan
Nov 16 '18 at 0:24
Ah, the
()
tripped up the replacement values. If you're dealing with accounting data, I think that might be a negative number... so you'll need to adjust accordingly. See: stackoverflow.com/questions/31521526/…– Evan
Nov 16 '18 at 0:24
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53329187%2fget-a-subset-of-a-data-frame-into-a-matrix%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Depending on your pandas version, pandas.pydata.org/pandas-docs/stable/generated/… or pandas.pydata.org/pandas-docs/stable/generated/… should do the trick. (You'll need to subset first, of course.)
– Evan
Nov 15 '18 at 23:17
@Evan Those did not really help me but thanks
– Snorrlaxxx
Nov 15 '18 at 23:22