Unsafe.As from byte array to ulong array
up vote
14
down vote
favorite
I'm currently looking at porting my metro hash implementon to use C#7 features, as several parts might profit from ref locals to improve performance.
The hash does the calculations on a ulong[4] array, but the result is a 16 byte array. Currently I'm copying the ulong array to the result byte buffer, but this takes a bit of time.
So i'm wondering if System.Runtime.CompilerServices.Unsafe is safe to use here:
var result = new byte[16];
ulong state = Unsafe.As<byte, ulong>(ref result);
ref var firstState = ref state[0];
ref var secondState = ref state[1];
ulong thirdState = 0;
ulong fourthState = 0;
The above code snippet means that I'm using the result buffer also for parts of my state calculations and not only for the final output.
My unit tests are successful and according to benchmarkdotnet skipping the block copy would result in a 20% performance increase, which is high enough for me to find out if it is correct to use it.
c# c#-7.0
add a comment |
up vote
14
down vote
favorite
I'm currently looking at porting my metro hash implementon to use C#7 features, as several parts might profit from ref locals to improve performance.
The hash does the calculations on a ulong[4] array, but the result is a 16 byte array. Currently I'm copying the ulong array to the result byte buffer, but this takes a bit of time.
So i'm wondering if System.Runtime.CompilerServices.Unsafe is safe to use here:
var result = new byte[16];
ulong state = Unsafe.As<byte, ulong>(ref result);
ref var firstState = ref state[0];
ref var secondState = ref state[1];
ulong thirdState = 0;
ulong fourthState = 0;
The above code snippet means that I'm using the result buffer also for parts of my state calculations and not only for the final output.
My unit tests are successful and according to benchmarkdotnet skipping the block copy would result in a 20% performance increase, which is high enough for me to find out if it is correct to use it.
c# c#-7.0
Welcome here and good first question!
– Patrick Hofman
Mar 9 '17 at 12:19
3
What you are doing is the old "trick" of casting through a struct (this one stackoverflow.com/a/35841815/613130)... If you check thestate.Lengthyou'll see that it is "wrong".
– xanatos
Mar 9 '17 at 12:19
2
Still very interesting library you have found :-)
– xanatos
Mar 9 '17 at 12:35
Thank you for answer, I know that explicit struct trick but actually didn't make the connection that it's actually the same as Unsafe.As<>.
– Tornhoof
Mar 9 '17 at 13:16
1
@Tornhoof TheUnsafe.As<>doesn't need that trick, but in the end it does the same thing. It reinterprets what is passed as the parameter to another type. In ILAsm it is very easy:ldarg.0; ret:-) Nothing to be done :-)
– xanatos
Mar 9 '17 at 13:21
add a comment |
up vote
14
down vote
favorite
up vote
14
down vote
favorite
I'm currently looking at porting my metro hash implementon to use C#7 features, as several parts might profit from ref locals to improve performance.
The hash does the calculations on a ulong[4] array, but the result is a 16 byte array. Currently I'm copying the ulong array to the result byte buffer, but this takes a bit of time.
So i'm wondering if System.Runtime.CompilerServices.Unsafe is safe to use here:
var result = new byte[16];
ulong state = Unsafe.As<byte, ulong>(ref result);
ref var firstState = ref state[0];
ref var secondState = ref state[1];
ulong thirdState = 0;
ulong fourthState = 0;
The above code snippet means that I'm using the result buffer also for parts of my state calculations and not only for the final output.
My unit tests are successful and according to benchmarkdotnet skipping the block copy would result in a 20% performance increase, which is high enough for me to find out if it is correct to use it.
c# c#-7.0
I'm currently looking at porting my metro hash implementon to use C#7 features, as several parts might profit from ref locals to improve performance.
The hash does the calculations on a ulong[4] array, but the result is a 16 byte array. Currently I'm copying the ulong array to the result byte buffer, but this takes a bit of time.
So i'm wondering if System.Runtime.CompilerServices.Unsafe is safe to use here:
var result = new byte[16];
ulong state = Unsafe.As<byte, ulong>(ref result);
ref var firstState = ref state[0];
ref var secondState = ref state[1];
ulong thirdState = 0;
ulong fourthState = 0;
The above code snippet means that I'm using the result buffer also for parts of my state calculations and not only for the final output.
My unit tests are successful and according to benchmarkdotnet skipping the block copy would result in a 20% performance increase, which is high enough for me to find out if it is correct to use it.
c# c#-7.0
c# c#-7.0
edited Jun 7 '17 at 6:18
Chandan Kumar
3,09732651
3,09732651
asked Mar 9 '17 at 12:10
Tornhoof
7124
7124
Welcome here and good first question!
– Patrick Hofman
Mar 9 '17 at 12:19
3
What you are doing is the old "trick" of casting through a struct (this one stackoverflow.com/a/35841815/613130)... If you check thestate.Lengthyou'll see that it is "wrong".
– xanatos
Mar 9 '17 at 12:19
2
Still very interesting library you have found :-)
– xanatos
Mar 9 '17 at 12:35
Thank you for answer, I know that explicit struct trick but actually didn't make the connection that it's actually the same as Unsafe.As<>.
– Tornhoof
Mar 9 '17 at 13:16
1
@Tornhoof TheUnsafe.As<>doesn't need that trick, but in the end it does the same thing. It reinterprets what is passed as the parameter to another type. In ILAsm it is very easy:ldarg.0; ret:-) Nothing to be done :-)
– xanatos
Mar 9 '17 at 13:21
add a comment |
Welcome here and good first question!
– Patrick Hofman
Mar 9 '17 at 12:19
3
What you are doing is the old "trick" of casting through a struct (this one stackoverflow.com/a/35841815/613130)... If you check thestate.Lengthyou'll see that it is "wrong".
– xanatos
Mar 9 '17 at 12:19
2
Still very interesting library you have found :-)
– xanatos
Mar 9 '17 at 12:35
Thank you for answer, I know that explicit struct trick but actually didn't make the connection that it's actually the same as Unsafe.As<>.
– Tornhoof
Mar 9 '17 at 13:16
1
@Tornhoof TheUnsafe.As<>doesn't need that trick, but in the end it does the same thing. It reinterprets what is passed as the parameter to another type. In ILAsm it is very easy:ldarg.0; ret:-) Nothing to be done :-)
– xanatos
Mar 9 '17 at 13:21
Welcome here and good first question!
– Patrick Hofman
Mar 9 '17 at 12:19
Welcome here and good first question!
– Patrick Hofman
Mar 9 '17 at 12:19
3
3
What you are doing is the old "trick" of casting through a struct (this one stackoverflow.com/a/35841815/613130)... If you check the
state.Length you'll see that it is "wrong".– xanatos
Mar 9 '17 at 12:19
What you are doing is the old "trick" of casting through a struct (this one stackoverflow.com/a/35841815/613130)... If you check the
state.Length you'll see that it is "wrong".– xanatos
Mar 9 '17 at 12:19
2
2
Still very interesting library you have found :-)
– xanatos
Mar 9 '17 at 12:35
Still very interesting library you have found :-)
– xanatos
Mar 9 '17 at 12:35
Thank you for answer, I know that explicit struct trick but actually didn't make the connection that it's actually the same as Unsafe.As<>.
– Tornhoof
Mar 9 '17 at 13:16
Thank you for answer, I know that explicit struct trick but actually didn't make the connection that it's actually the same as Unsafe.As<>.
– Tornhoof
Mar 9 '17 at 13:16
1
1
@Tornhoof The
Unsafe.As<> doesn't need that trick, but in the end it does the same thing. It reinterprets what is passed as the parameter to another type. In ILAsm it is very easy: ldarg.0; ret :-) Nothing to be done :-)– xanatos
Mar 9 '17 at 13:21
@Tornhoof The
Unsafe.As<> doesn't need that trick, but in the end it does the same thing. It reinterprets what is passed as the parameter to another type. In ILAsm it is very easy: ldarg.0; ret :-) Nothing to be done :-)– xanatos
Mar 9 '17 at 13:21
add a comment |
3 Answers
3
active
oldest
votes
up vote
4
down vote
In current .NET terms, this would be a good fit for Span<T>:
Span<byte> result = new byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
This enforces lengths etc, while having good JIT behaviour and not requiring unsafe. You can even stackalloc the original buffer (from C# 7.2 onwards):
Span<byte> result = stackalloc byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
Note that Span<T> gets the length change correct; it is also trivial to cast into a Span<Vector<T>> if you want to use SIMD for hardware acceleration.
add a comment |
up vote
1
down vote
C# supports "fixed buffers", here's the kind of thing we can do:
public unsafe struct Bytes
{
public fixed byte bytes[16];
}
then
public unsafe static Bytes Convert (long longs)
{
fixed (long * longs_ptr = longs)
return *((Bytes*)(longs_ptr));
}
Try it. (1D arrays of primitive types in C# are always stored as a contiguous block of memory which is why taking the address of the (managed) arrays is fine).
You could also even return the pointer for more speed:
public unsafe static Bytes * Convert (long longs)
{
fixed (long * longs_ptr = longs)
return ((Bytes*)(longs_ptr));
}
and manipulate/access the bytes as you want.
var s = Convert(longs);
var b = s->bytes[0];
1
"You could also" - no! don't do that; as soon as you leave thefixedblock, the pointer is unreliable; in reality it will rarely move, but: it can (GC); never pass a pointer outwards from afixedblock (inwards is fine)
– Marc Gravell♦
Nov 11 at 21:56
add a comment |
up vote
1
down vote
What you're doing seems fine, just be careful because there's nothing to stop you from doing this:
byte x = new byte[16];
long y = Unsafe.As<byte, long>(ref x);
Console.WriteLine(y.Length); // still 16
for (int i = 0; i < y.Length; i++)
Console.WriteLine(y[i]); // reads random memory from your program, could cause crash
1
re the length problem:Span<T>could help here - see my answer
– Marc Gravell♦
Nov 11 at 22:01
@MarcGravell Yeah that's the safer option, though I believe a touch slower because of the length conversions and such. I haven't benchmarked the difference though. As long as he's careful he might be able to eek a touch more perf out of the way he's doing it now. I'm completely in love the new span/memory/unsafe stuff, it makes me want to go and rewrite everything.
– Mike Marynowski
Nov 12 at 23:11
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
In current .NET terms, this would be a good fit for Span<T>:
Span<byte> result = new byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
This enforces lengths etc, while having good JIT behaviour and not requiring unsafe. You can even stackalloc the original buffer (from C# 7.2 onwards):
Span<byte> result = stackalloc byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
Note that Span<T> gets the length change correct; it is also trivial to cast into a Span<Vector<T>> if you want to use SIMD for hardware acceleration.
add a comment |
up vote
4
down vote
In current .NET terms, this would be a good fit for Span<T>:
Span<byte> result = new byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
This enforces lengths etc, while having good JIT behaviour and not requiring unsafe. You can even stackalloc the original buffer (from C# 7.2 onwards):
Span<byte> result = stackalloc byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
Note that Span<T> gets the length change correct; it is also trivial to cast into a Span<Vector<T>> if you want to use SIMD for hardware acceleration.
add a comment |
up vote
4
down vote
up vote
4
down vote
In current .NET terms, this would be a good fit for Span<T>:
Span<byte> result = new byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
This enforces lengths etc, while having good JIT behaviour and not requiring unsafe. You can even stackalloc the original buffer (from C# 7.2 onwards):
Span<byte> result = stackalloc byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
Note that Span<T> gets the length change correct; it is also trivial to cast into a Span<Vector<T>> if you want to use SIMD for hardware acceleration.
In current .NET terms, this would be a good fit for Span<T>:
Span<byte> result = new byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
This enforces lengths etc, while having good JIT behaviour and not requiring unsafe. You can even stackalloc the original buffer (from C# 7.2 onwards):
Span<byte> result = stackalloc byte[16];
Span<ulong> state = MemoryMarshal.Cast<byte, ulong>(result);
Note that Span<T> gets the length change correct; it is also trivial to cast into a Span<Vector<T>> if you want to use SIMD for hardware acceleration.
edited Nov 11 at 22:09
answered Nov 11 at 22:01
Marc Gravell♦
774k19021242538
774k19021242538
add a comment |
add a comment |
up vote
1
down vote
C# supports "fixed buffers", here's the kind of thing we can do:
public unsafe struct Bytes
{
public fixed byte bytes[16];
}
then
public unsafe static Bytes Convert (long longs)
{
fixed (long * longs_ptr = longs)
return *((Bytes*)(longs_ptr));
}
Try it. (1D arrays of primitive types in C# are always stored as a contiguous block of memory which is why taking the address of the (managed) arrays is fine).
You could also even return the pointer for more speed:
public unsafe static Bytes * Convert (long longs)
{
fixed (long * longs_ptr = longs)
return ((Bytes*)(longs_ptr));
}
and manipulate/access the bytes as you want.
var s = Convert(longs);
var b = s->bytes[0];
1
"You could also" - no! don't do that; as soon as you leave thefixedblock, the pointer is unreliable; in reality it will rarely move, but: it can (GC); never pass a pointer outwards from afixedblock (inwards is fine)
– Marc Gravell♦
Nov 11 at 21:56
add a comment |
up vote
1
down vote
C# supports "fixed buffers", here's the kind of thing we can do:
public unsafe struct Bytes
{
public fixed byte bytes[16];
}
then
public unsafe static Bytes Convert (long longs)
{
fixed (long * longs_ptr = longs)
return *((Bytes*)(longs_ptr));
}
Try it. (1D arrays of primitive types in C# are always stored as a contiguous block of memory which is why taking the address of the (managed) arrays is fine).
You could also even return the pointer for more speed:
public unsafe static Bytes * Convert (long longs)
{
fixed (long * longs_ptr = longs)
return ((Bytes*)(longs_ptr));
}
and manipulate/access the bytes as you want.
var s = Convert(longs);
var b = s->bytes[0];
1
"You could also" - no! don't do that; as soon as you leave thefixedblock, the pointer is unreliable; in reality it will rarely move, but: it can (GC); never pass a pointer outwards from afixedblock (inwards is fine)
– Marc Gravell♦
Nov 11 at 21:56
add a comment |
up vote
1
down vote
up vote
1
down vote
C# supports "fixed buffers", here's the kind of thing we can do:
public unsafe struct Bytes
{
public fixed byte bytes[16];
}
then
public unsafe static Bytes Convert (long longs)
{
fixed (long * longs_ptr = longs)
return *((Bytes*)(longs_ptr));
}
Try it. (1D arrays of primitive types in C# are always stored as a contiguous block of memory which is why taking the address of the (managed) arrays is fine).
You could also even return the pointer for more speed:
public unsafe static Bytes * Convert (long longs)
{
fixed (long * longs_ptr = longs)
return ((Bytes*)(longs_ptr));
}
and manipulate/access the bytes as you want.
var s = Convert(longs);
var b = s->bytes[0];
C# supports "fixed buffers", here's the kind of thing we can do:
public unsafe struct Bytes
{
public fixed byte bytes[16];
}
then
public unsafe static Bytes Convert (long longs)
{
fixed (long * longs_ptr = longs)
return *((Bytes*)(longs_ptr));
}
Try it. (1D arrays of primitive types in C# are always stored as a contiguous block of memory which is why taking the address of the (managed) arrays is fine).
You could also even return the pointer for more speed:
public unsafe static Bytes * Convert (long longs)
{
fixed (long * longs_ptr = longs)
return ((Bytes*)(longs_ptr));
}
and manipulate/access the bytes as you want.
var s = Convert(longs);
var b = s->bytes[0];
edited Jun 16 '17 at 21:45
answered Jun 16 '17 at 21:39
Hugh
42246
42246
1
"You could also" - no! don't do that; as soon as you leave thefixedblock, the pointer is unreliable; in reality it will rarely move, but: it can (GC); never pass a pointer outwards from afixedblock (inwards is fine)
– Marc Gravell♦
Nov 11 at 21:56
add a comment |
1
"You could also" - no! don't do that; as soon as you leave thefixedblock, the pointer is unreliable; in reality it will rarely move, but: it can (GC); never pass a pointer outwards from afixedblock (inwards is fine)
– Marc Gravell♦
Nov 11 at 21:56
1
1
"You could also" - no! don't do that; as soon as you leave the
fixed block, the pointer is unreliable; in reality it will rarely move, but: it can (GC); never pass a pointer outwards from a fixed block (inwards is fine)– Marc Gravell♦
Nov 11 at 21:56
"You could also" - no! don't do that; as soon as you leave the
fixed block, the pointer is unreliable; in reality it will rarely move, but: it can (GC); never pass a pointer outwards from a fixed block (inwards is fine)– Marc Gravell♦
Nov 11 at 21:56
add a comment |
up vote
1
down vote
What you're doing seems fine, just be careful because there's nothing to stop you from doing this:
byte x = new byte[16];
long y = Unsafe.As<byte, long>(ref x);
Console.WriteLine(y.Length); // still 16
for (int i = 0; i < y.Length; i++)
Console.WriteLine(y[i]); // reads random memory from your program, could cause crash
1
re the length problem:Span<T>could help here - see my answer
– Marc Gravell♦
Nov 11 at 22:01
@MarcGravell Yeah that's the safer option, though I believe a touch slower because of the length conversions and such. I haven't benchmarked the difference though. As long as he's careful he might be able to eek a touch more perf out of the way he's doing it now. I'm completely in love the new span/memory/unsafe stuff, it makes me want to go and rewrite everything.
– Mike Marynowski
Nov 12 at 23:11
add a comment |
up vote
1
down vote
What you're doing seems fine, just be careful because there's nothing to stop you from doing this:
byte x = new byte[16];
long y = Unsafe.As<byte, long>(ref x);
Console.WriteLine(y.Length); // still 16
for (int i = 0; i < y.Length; i++)
Console.WriteLine(y[i]); // reads random memory from your program, could cause crash
1
re the length problem:Span<T>could help here - see my answer
– Marc Gravell♦
Nov 11 at 22:01
@MarcGravell Yeah that's the safer option, though I believe a touch slower because of the length conversions and such. I haven't benchmarked the difference though. As long as he's careful he might be able to eek a touch more perf out of the way he's doing it now. I'm completely in love the new span/memory/unsafe stuff, it makes me want to go and rewrite everything.
– Mike Marynowski
Nov 12 at 23:11
add a comment |
up vote
1
down vote
up vote
1
down vote
What you're doing seems fine, just be careful because there's nothing to stop you from doing this:
byte x = new byte[16];
long y = Unsafe.As<byte, long>(ref x);
Console.WriteLine(y.Length); // still 16
for (int i = 0; i < y.Length; i++)
Console.WriteLine(y[i]); // reads random memory from your program, could cause crash
What you're doing seems fine, just be careful because there's nothing to stop you from doing this:
byte x = new byte[16];
long y = Unsafe.As<byte, long>(ref x);
Console.WriteLine(y.Length); // still 16
for (int i = 0; i < y.Length; i++)
Console.WriteLine(y[i]); // reads random memory from your program, could cause crash
answered Nov 11 at 21:06
Mike Marynowski
1,8061126
1,8061126
1
re the length problem:Span<T>could help here - see my answer
– Marc Gravell♦
Nov 11 at 22:01
@MarcGravell Yeah that's the safer option, though I believe a touch slower because of the length conversions and such. I haven't benchmarked the difference though. As long as he's careful he might be able to eek a touch more perf out of the way he's doing it now. I'm completely in love the new span/memory/unsafe stuff, it makes me want to go and rewrite everything.
– Mike Marynowski
Nov 12 at 23:11
add a comment |
1
re the length problem:Span<T>could help here - see my answer
– Marc Gravell♦
Nov 11 at 22:01
@MarcGravell Yeah that's the safer option, though I believe a touch slower because of the length conversions and such. I haven't benchmarked the difference though. As long as he's careful he might be able to eek a touch more perf out of the way he's doing it now. I'm completely in love the new span/memory/unsafe stuff, it makes me want to go and rewrite everything.
– Mike Marynowski
Nov 12 at 23:11
1
1
re the length problem:
Span<T> could help here - see my answer– Marc Gravell♦
Nov 11 at 22:01
re the length problem:
Span<T> could help here - see my answer– Marc Gravell♦
Nov 11 at 22:01
@MarcGravell Yeah that's the safer option, though I believe a touch slower because of the length conversions and such. I haven't benchmarked the difference though. As long as he's careful he might be able to eek a touch more perf out of the way he's doing it now. I'm completely in love the new span/memory/unsafe stuff, it makes me want to go and rewrite everything.
– Mike Marynowski
Nov 12 at 23:11
@MarcGravell Yeah that's the safer option, though I believe a touch slower because of the length conversions and such. I haven't benchmarked the difference though. As long as he's careful he might be able to eek a touch more perf out of the way he's doing it now. I'm completely in love the new span/memory/unsafe stuff, it makes me want to go and rewrite everything.
– Mike Marynowski
Nov 12 at 23:11
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f42695067%2funsafe-as-from-byte-array-to-ulong-array%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Welcome here and good first question!
– Patrick Hofman
Mar 9 '17 at 12:19
3
What you are doing is the old "trick" of casting through a struct (this one stackoverflow.com/a/35841815/613130)... If you check the
state.Lengthyou'll see that it is "wrong".– xanatos
Mar 9 '17 at 12:19
2
Still very interesting library you have found :-)
– xanatos
Mar 9 '17 at 12:35
Thank you for answer, I know that explicit struct trick but actually didn't make the connection that it's actually the same as Unsafe.As<>.
– Tornhoof
Mar 9 '17 at 13:16
1
@Tornhoof The
Unsafe.As<>doesn't need that trick, but in the end it does the same thing. It reinterprets what is passed as the parameter to another type. In ILAsm it is very easy:ldarg.0; ret:-) Nothing to be done :-)– xanatos
Mar 9 '17 at 13:21