Strange behavior of Numpy `where` clause [duplicate]
This question already has an answer here:
What is the default of numpy functions, with where=False?
1 answer
I see a strange behavior with ufunc where clause for Numpy 1.15.3.
In [1]: import numpy as np
In [2]: x = np.array([[1,2],[3,4]])
In [3]: y = np.ones(x.shape) * 2
In [4]: print(x, "n", y)
[[1 2]
[3 4]]
[[2. 2.]
[2. 2.]]
In [5]: np.add(x, y, where=x==3)
Out[5]:
array([[2., 2.], #<=========== where do these 2s come from???
[5., 2.]])
In [6]: np.add(x, y, where=x==3, out=np.zeros(x.shape))
Out[6]:
array([[0., 0.],
[5., 0.]])
In [7]: np.add(x, y, where=x==3, out=np.ones(x.shape))
Out[7]:
array([[1., 1.],
[5., 1.]])
In [8]: np.add(x, y, where=x==3)
Out[8]:
array([[1., 1.], # <========= it seems these 1s are remembered from last computation.
[5., 1.]])
ADD1
It seems I can only get the rational result with the out parameter.
Below is without the out parameter:
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
x = np.linspace(-2,2,60)
y = np.linspace(-2,2,60)
xx, yy = np.meshgrid(x,y)
r= np.ones((60,60), dtype=float) * 2
z = np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0) # <==== HERE!!
surf = ax.plot_surface(xx, yy, z, cmap="viridis")
This generates a ridiculous image:

If I add the out parameter as below, everything works fine.
z = np.zeros(xx.shape)
np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0, out=z)

numpy pythonanywhere numpy-broadcasting
marked as duplicate by Warren Weckesser
StackExchange.ready(function() {
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function() {
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function() {
$hover.showInfoMessage('', {
messageElement: $msg.clone().show(),
transient: false,
position: { my: 'bottom left', at: 'top center', offsetTop: -7 },
dismissable: false,
relativeToBody: true
});
},
function() {
StackExchange.helpers.removeMessages();
}
);
});
});
Nov 15 '18 at 5:27
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
add a comment |
This question already has an answer here:
What is the default of numpy functions, with where=False?
1 answer
I see a strange behavior with ufunc where clause for Numpy 1.15.3.
In [1]: import numpy as np
In [2]: x = np.array([[1,2],[3,4]])
In [3]: y = np.ones(x.shape) * 2
In [4]: print(x, "n", y)
[[1 2]
[3 4]]
[[2. 2.]
[2. 2.]]
In [5]: np.add(x, y, where=x==3)
Out[5]:
array([[2., 2.], #<=========== where do these 2s come from???
[5., 2.]])
In [6]: np.add(x, y, where=x==3, out=np.zeros(x.shape))
Out[6]:
array([[0., 0.],
[5., 0.]])
In [7]: np.add(x, y, where=x==3, out=np.ones(x.shape))
Out[7]:
array([[1., 1.],
[5., 1.]])
In [8]: np.add(x, y, where=x==3)
Out[8]:
array([[1., 1.], # <========= it seems these 1s are remembered from last computation.
[5., 1.]])
ADD1
It seems I can only get the rational result with the out parameter.
Below is without the out parameter:
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
x = np.linspace(-2,2,60)
y = np.linspace(-2,2,60)
xx, yy = np.meshgrid(x,y)
r= np.ones((60,60), dtype=float) * 2
z = np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0) # <==== HERE!!
surf = ax.plot_surface(xx, yy, z, cmap="viridis")
This generates a ridiculous image:

If I add the out parameter as below, everything works fine.
z = np.zeros(xx.shape)
np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0, out=z)

numpy pythonanywhere numpy-broadcasting
marked as duplicate by Warren Weckesser
StackExchange.ready(function() {
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function() {
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function() {
$hover.showInfoMessage('', {
messageElement: $msg.clone().show(),
transient: false,
position: { my: 'bottom left', at: 'top center', offsetTop: -7 },
dismissable: false,
relativeToBody: true
});
},
function() {
StackExchange.helpers.removeMessages();
}
);
});
});
Nov 15 '18 at 5:27
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
"where do these 2s come from???" See @tel's answer. The 2's might be from a temporary array being created, freed, and then the same memory being reused forout.
– Warren Weckesser
Nov 15 '18 at 5:19
This question looks like a duplicate of stackoverflow.com/questions/45543505/…; also see stackoverflow.com/questions/49459985/…
– Warren Weckesser
Nov 15 '18 at 5:24
add a comment |
This question already has an answer here:
What is the default of numpy functions, with where=False?
1 answer
I see a strange behavior with ufunc where clause for Numpy 1.15.3.
In [1]: import numpy as np
In [2]: x = np.array([[1,2],[3,4]])
In [3]: y = np.ones(x.shape) * 2
In [4]: print(x, "n", y)
[[1 2]
[3 4]]
[[2. 2.]
[2. 2.]]
In [5]: np.add(x, y, where=x==3)
Out[5]:
array([[2., 2.], #<=========== where do these 2s come from???
[5., 2.]])
In [6]: np.add(x, y, where=x==3, out=np.zeros(x.shape))
Out[6]:
array([[0., 0.],
[5., 0.]])
In [7]: np.add(x, y, where=x==3, out=np.ones(x.shape))
Out[7]:
array([[1., 1.],
[5., 1.]])
In [8]: np.add(x, y, where=x==3)
Out[8]:
array([[1., 1.], # <========= it seems these 1s are remembered from last computation.
[5., 1.]])
ADD1
It seems I can only get the rational result with the out parameter.
Below is without the out parameter:
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
x = np.linspace(-2,2,60)
y = np.linspace(-2,2,60)
xx, yy = np.meshgrid(x,y)
r= np.ones((60,60), dtype=float) * 2
z = np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0) # <==== HERE!!
surf = ax.plot_surface(xx, yy, z, cmap="viridis")
This generates a ridiculous image:

If I add the out parameter as below, everything works fine.
z = np.zeros(xx.shape)
np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0, out=z)

numpy pythonanywhere numpy-broadcasting
This question already has an answer here:
What is the default of numpy functions, with where=False?
1 answer
I see a strange behavior with ufunc where clause for Numpy 1.15.3.
In [1]: import numpy as np
In [2]: x = np.array([[1,2],[3,4]])
In [3]: y = np.ones(x.shape) * 2
In [4]: print(x, "n", y)
[[1 2]
[3 4]]
[[2. 2.]
[2. 2.]]
In [5]: np.add(x, y, where=x==3)
Out[5]:
array([[2., 2.], #<=========== where do these 2s come from???
[5., 2.]])
In [6]: np.add(x, y, where=x==3, out=np.zeros(x.shape))
Out[6]:
array([[0., 0.],
[5., 0.]])
In [7]: np.add(x, y, where=x==3, out=np.ones(x.shape))
Out[7]:
array([[1., 1.],
[5., 1.]])
In [8]: np.add(x, y, where=x==3)
Out[8]:
array([[1., 1.], # <========= it seems these 1s are remembered from last computation.
[5., 1.]])
ADD1
It seems I can only get the rational result with the out parameter.
Below is without the out parameter:
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
x = np.linspace(-2,2,60)
y = np.linspace(-2,2,60)
xx, yy = np.meshgrid(x,y)
r= np.ones((60,60), dtype=float) * 2
z = np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0) # <==== HERE!!
surf = ax.plot_surface(xx, yy, z, cmap="viridis")
This generates a ridiculous image:

If I add the out parameter as below, everything works fine.
z = np.zeros(xx.shape)
np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0, out=z)

This question already has an answer here:
What is the default of numpy functions, with where=False?
1 answer
numpy pythonanywhere numpy-broadcasting
numpy pythonanywhere numpy-broadcasting
edited Nov 15 '18 at 5:20
smwikipedia
asked Nov 15 '18 at 5:04
smwikipediasmwikipedia
21.6k64215370
21.6k64215370
marked as duplicate by Warren Weckesser
StackExchange.ready(function() {
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function() {
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function() {
$hover.showInfoMessage('', {
messageElement: $msg.clone().show(),
transient: false,
position: { my: 'bottom left', at: 'top center', offsetTop: -7 },
dismissable: false,
relativeToBody: true
});
},
function() {
StackExchange.helpers.removeMessages();
}
);
});
});
Nov 15 '18 at 5:27
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
marked as duplicate by Warren Weckesser
StackExchange.ready(function() {
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function() {
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function() {
$hover.showInfoMessage('', {
messageElement: $msg.clone().show(),
transient: false,
position: { my: 'bottom left', at: 'top center', offsetTop: -7 },
dismissable: false,
relativeToBody: true
});
},
function() {
StackExchange.helpers.removeMessages();
}
);
});
});
Nov 15 '18 at 5:27
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
"where do these 2s come from???" See @tel's answer. The 2's might be from a temporary array being created, freed, and then the same memory being reused forout.
– Warren Weckesser
Nov 15 '18 at 5:19
This question looks like a duplicate of stackoverflow.com/questions/45543505/…; also see stackoverflow.com/questions/49459985/…
– Warren Weckesser
Nov 15 '18 at 5:24
add a comment |
"where do these 2s come from???" See @tel's answer. The 2's might be from a temporary array being created, freed, and then the same memory being reused forout.
– Warren Weckesser
Nov 15 '18 at 5:19
This question looks like a duplicate of stackoverflow.com/questions/45543505/…; also see stackoverflow.com/questions/49459985/…
– Warren Weckesser
Nov 15 '18 at 5:24
"where do these 2s come from???" See @tel's answer. The 2's might be from a temporary array being created, freed, and then the same memory being reused for
out.– Warren Weckesser
Nov 15 '18 at 5:19
"where do these 2s come from???" See @tel's answer. The 2's might be from a temporary array being created, freed, and then the same memory being reused for
out.– Warren Weckesser
Nov 15 '18 at 5:19
This question looks like a duplicate of stackoverflow.com/questions/45543505/…; also see stackoverflow.com/questions/49459985/…
– Warren Weckesser
Nov 15 '18 at 5:24
This question looks like a duplicate of stackoverflow.com/questions/45543505/…; also see stackoverflow.com/questions/49459985/…
– Warren Weckesser
Nov 15 '18 at 5:24
add a comment |
1 Answer
1
active
oldest
votes
You're ending up with junk data in your outputs due to the use of where. The fix, as you've said, is to initialize your own output and pass that to out.
From the docs about the out arg:
If ‘out’ is None (the default), a uninitialized return array is created. The output array is then filled with the results of the ufunc in the places that the broadcast ‘where’ is True. If ‘where’ is the scalar True (the default), then this corresponds to the entire output being filled. Note that outputs not explicitly filled are left with their uninitialized values.
So the values of out that you skip over (ie the indices in which where is False) are just left with whatever value was in them previously. That's why it looks like numpy is "remebering" values from previous computations, like the 1s at the end of your first example code block.
As @WarrenWeckesser points out in his comment, this also implies that the same memory block is being reused for the output when out is left blank, at least in some cases. Interestingly, you can alter the results you were getting by assigning each output to a variable:
x = np.array([[1,2],[3,4]])
y = np.ones(x.shape) * 2
arr0 = np.add(x, y, where=x==3)
arr1 = np.add(x, y, where=x==3, out=np.zeros(x.shape))
arr2 = np.add(x, y, where=x==3, out=np.ones(x.shape))
arr3 = np.add(x, y, where=x==3)
print(arr3)
Now you can clearly see the junk data in the output:
[[-2.68156159e+154 -2.68156159e+154]
[ 5.00000000e+000 2.82470645e-309]]
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You're ending up with junk data in your outputs due to the use of where. The fix, as you've said, is to initialize your own output and pass that to out.
From the docs about the out arg:
If ‘out’ is None (the default), a uninitialized return array is created. The output array is then filled with the results of the ufunc in the places that the broadcast ‘where’ is True. If ‘where’ is the scalar True (the default), then this corresponds to the entire output being filled. Note that outputs not explicitly filled are left with their uninitialized values.
So the values of out that you skip over (ie the indices in which where is False) are just left with whatever value was in them previously. That's why it looks like numpy is "remebering" values from previous computations, like the 1s at the end of your first example code block.
As @WarrenWeckesser points out in his comment, this also implies that the same memory block is being reused for the output when out is left blank, at least in some cases. Interestingly, you can alter the results you were getting by assigning each output to a variable:
x = np.array([[1,2],[3,4]])
y = np.ones(x.shape) * 2
arr0 = np.add(x, y, where=x==3)
arr1 = np.add(x, y, where=x==3, out=np.zeros(x.shape))
arr2 = np.add(x, y, where=x==3, out=np.ones(x.shape))
arr3 = np.add(x, y, where=x==3)
print(arr3)
Now you can clearly see the junk data in the output:
[[-2.68156159e+154 -2.68156159e+154]
[ 5.00000000e+000 2.82470645e-309]]
add a comment |
You're ending up with junk data in your outputs due to the use of where. The fix, as you've said, is to initialize your own output and pass that to out.
From the docs about the out arg:
If ‘out’ is None (the default), a uninitialized return array is created. The output array is then filled with the results of the ufunc in the places that the broadcast ‘where’ is True. If ‘where’ is the scalar True (the default), then this corresponds to the entire output being filled. Note that outputs not explicitly filled are left with their uninitialized values.
So the values of out that you skip over (ie the indices in which where is False) are just left with whatever value was in them previously. That's why it looks like numpy is "remebering" values from previous computations, like the 1s at the end of your first example code block.
As @WarrenWeckesser points out in his comment, this also implies that the same memory block is being reused for the output when out is left blank, at least in some cases. Interestingly, you can alter the results you were getting by assigning each output to a variable:
x = np.array([[1,2],[3,4]])
y = np.ones(x.shape) * 2
arr0 = np.add(x, y, where=x==3)
arr1 = np.add(x, y, where=x==3, out=np.zeros(x.shape))
arr2 = np.add(x, y, where=x==3, out=np.ones(x.shape))
arr3 = np.add(x, y, where=x==3)
print(arr3)
Now you can clearly see the junk data in the output:
[[-2.68156159e+154 -2.68156159e+154]
[ 5.00000000e+000 2.82470645e-309]]
add a comment |
You're ending up with junk data in your outputs due to the use of where. The fix, as you've said, is to initialize your own output and pass that to out.
From the docs about the out arg:
If ‘out’ is None (the default), a uninitialized return array is created. The output array is then filled with the results of the ufunc in the places that the broadcast ‘where’ is True. If ‘where’ is the scalar True (the default), then this corresponds to the entire output being filled. Note that outputs not explicitly filled are left with their uninitialized values.
So the values of out that you skip over (ie the indices in which where is False) are just left with whatever value was in them previously. That's why it looks like numpy is "remebering" values from previous computations, like the 1s at the end of your first example code block.
As @WarrenWeckesser points out in his comment, this also implies that the same memory block is being reused for the output when out is left blank, at least in some cases. Interestingly, you can alter the results you were getting by assigning each output to a variable:
x = np.array([[1,2],[3,4]])
y = np.ones(x.shape) * 2
arr0 = np.add(x, y, where=x==3)
arr1 = np.add(x, y, where=x==3, out=np.zeros(x.shape))
arr2 = np.add(x, y, where=x==3, out=np.ones(x.shape))
arr3 = np.add(x, y, where=x==3)
print(arr3)
Now you can clearly see the junk data in the output:
[[-2.68156159e+154 -2.68156159e+154]
[ 5.00000000e+000 2.82470645e-309]]
You're ending up with junk data in your outputs due to the use of where. The fix, as you've said, is to initialize your own output and pass that to out.
From the docs about the out arg:
If ‘out’ is None (the default), a uninitialized return array is created. The output array is then filled with the results of the ufunc in the places that the broadcast ‘where’ is True. If ‘where’ is the scalar True (the default), then this corresponds to the entire output being filled. Note that outputs not explicitly filled are left with their uninitialized values.
So the values of out that you skip over (ie the indices in which where is False) are just left with whatever value was in them previously. That's why it looks like numpy is "remebering" values from previous computations, like the 1s at the end of your first example code block.
As @WarrenWeckesser points out in his comment, this also implies that the same memory block is being reused for the output when out is left blank, at least in some cases. Interestingly, you can alter the results you were getting by assigning each output to a variable:
x = np.array([[1,2],[3,4]])
y = np.ones(x.shape) * 2
arr0 = np.add(x, y, where=x==3)
arr1 = np.add(x, y, where=x==3, out=np.zeros(x.shape))
arr2 = np.add(x, y, where=x==3, out=np.ones(x.shape))
arr3 = np.add(x, y, where=x==3)
print(arr3)
Now you can clearly see the junk data in the output:
[[-2.68156159e+154 -2.68156159e+154]
[ 5.00000000e+000 2.82470645e-309]]
edited Nov 15 '18 at 5:45
answered Nov 15 '18 at 5:17
teltel
7,41621431
7,41621431
add a comment |
add a comment |
"where do these 2s come from???" See @tel's answer. The 2's might be from a temporary array being created, freed, and then the same memory being reused for
out.– Warren Weckesser
Nov 15 '18 at 5:19
This question looks like a duplicate of stackoverflow.com/questions/45543505/…; also see stackoverflow.com/questions/49459985/…
– Warren Weckesser
Nov 15 '18 at 5:24