ASCII encoding for string in GO
In Ruby, you can encode string to ASCII as follows:
str.force_encoding('ASCII')
How can we achieve the same in Go?
go encoding ascii
add a comment |
In Ruby, you can encode string to ASCII as follows:
str.force_encoding('ASCII')
How can we achieve the same in Go?
go encoding ascii
2
What do you expect to happen if the string contains a byte outside the range[0,127]
?
– maerics
Nov 13 '18 at 20:17
add a comment |
In Ruby, you can encode string to ASCII as follows:
str.force_encoding('ASCII')
How can we achieve the same in Go?
go encoding ascii
In Ruby, you can encode string to ASCII as follows:
str.force_encoding('ASCII')
How can we achieve the same in Go?
go encoding ascii
go encoding ascii
edited Nov 13 '18 at 18:29
Madhur Bhaiya
19.6k62236
19.6k62236
asked Nov 13 '18 at 18:17
Pritish ShahPritish Shah
3291722
3291722
2
What do you expect to happen if the string contains a byte outside the range[0,127]
?
– maerics
Nov 13 '18 at 20:17
add a comment |
2
What do you expect to happen if the string contains a byte outside the range[0,127]
?
– maerics
Nov 13 '18 at 20:17
2
2
What do you expect to happen if the string contains a byte outside the range
[0,127]
?– maerics
Nov 13 '18 at 20:17
What do you expect to happen if the string contains a byte outside the range
[0,127]
?– maerics
Nov 13 '18 at 20:17
add a comment |
2 Answers
2
active
oldest
votes
strconv.QuoteToASCII
QuoteToASCII returns a double-quoted Go string literal representing s. The returned string uses Go escape sequences (t, n, xFF, u0100) for non-ASCII characters and non-printable characters as defined by IsPrint.
Or if you want an array of ascii codes, you could do
import "encoding/ascii85"
dst := make(byte, 25, 25)
dst2 := make(byte, 25, 25)
ascii85.Encode(dst, byte("Hello, playground"))
fmt.Println(dst)
ascii85.Decode(dst2, dst, false)
fmt.Println(string(dst2))
https://play.golang.org/p/gLEuWAGglJV
add a comment |
A simple version that omits invalid runes could look like this:
func forceASCII(s string) string {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
}
}
return string(rs)
}
// forceASCII("Hello, World!") // => "Hello, World!"
// forceASCII("Hello, 世界!") // => "Hello, !"
// forceASCII("Привет") // => ""
But what if you want special behavior if the target UTF-8 string contains any characters outside the ASCII character range of [0,127]
?
You could write a function that handles various cases by extracting a function argument which takes the invalid-ASCII rune and returns a string replacement or error.
For example (Go Playground):
func forceASCII(s string, replacer func(rune) (string, error)) (string, error) {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
} else {
replacement, err := replacer(r)
if err != nil {
return "", err
}
rs = append(rs, rune(replacement)...)
}
}
return string(rs), nil
}
func main() {
replacers := func(r rune) (string, error){
// omit invalid runes
func(_ rune) (string, error) { return "", nil },
// replace with question marks
func(_ rune) (string, error) { return "?", nil },
// abort with error */
func(r rune) (string, error) { return "", fmt.Errorf("invalid rune 0x%x", r) },
}
ss := string{"Hello, World!", "Hello, 世界!"}
for _, s := range ss {
for _, r := range replacers {
ascii, err := forceASCII(s, r)
fmt.Printf("OK: %q → %q, err=%vn", s, ascii, err)
}
}
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, 世界!" → "Hello, !", err=<nil>
// OK: "Hello, 世界!" → "Hello, ??!", err=<nil>
// OK: "Hello, 世界!" → "", err=invalid rune 0x4e16
}
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53287230%2fascii-encoding-for-string-in-go%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
strconv.QuoteToASCII
QuoteToASCII returns a double-quoted Go string literal representing s. The returned string uses Go escape sequences (t, n, xFF, u0100) for non-ASCII characters and non-printable characters as defined by IsPrint.
Or if you want an array of ascii codes, you could do
import "encoding/ascii85"
dst := make(byte, 25, 25)
dst2 := make(byte, 25, 25)
ascii85.Encode(dst, byte("Hello, playground"))
fmt.Println(dst)
ascii85.Decode(dst2, dst, false)
fmt.Println(string(dst2))
https://play.golang.org/p/gLEuWAGglJV
add a comment |
strconv.QuoteToASCII
QuoteToASCII returns a double-quoted Go string literal representing s. The returned string uses Go escape sequences (t, n, xFF, u0100) for non-ASCII characters and non-printable characters as defined by IsPrint.
Or if you want an array of ascii codes, you could do
import "encoding/ascii85"
dst := make(byte, 25, 25)
dst2 := make(byte, 25, 25)
ascii85.Encode(dst, byte("Hello, playground"))
fmt.Println(dst)
ascii85.Decode(dst2, dst, false)
fmt.Println(string(dst2))
https://play.golang.org/p/gLEuWAGglJV
add a comment |
strconv.QuoteToASCII
QuoteToASCII returns a double-quoted Go string literal representing s. The returned string uses Go escape sequences (t, n, xFF, u0100) for non-ASCII characters and non-printable characters as defined by IsPrint.
Or if you want an array of ascii codes, you could do
import "encoding/ascii85"
dst := make(byte, 25, 25)
dst2 := make(byte, 25, 25)
ascii85.Encode(dst, byte("Hello, playground"))
fmt.Println(dst)
ascii85.Decode(dst2, dst, false)
fmt.Println(string(dst2))
https://play.golang.org/p/gLEuWAGglJV
strconv.QuoteToASCII
QuoteToASCII returns a double-quoted Go string literal representing s. The returned string uses Go escape sequences (t, n, xFF, u0100) for non-ASCII characters and non-printable characters as defined by IsPrint.
Or if you want an array of ascii codes, you could do
import "encoding/ascii85"
dst := make(byte, 25, 25)
dst2 := make(byte, 25, 25)
ascii85.Encode(dst, byte("Hello, playground"))
fmt.Println(dst)
ascii85.Decode(dst2, dst, false)
fmt.Println(string(dst2))
https://play.golang.org/p/gLEuWAGglJV
answered Nov 13 '18 at 18:29
davedave
34.8k13564
34.8k13564
add a comment |
add a comment |
A simple version that omits invalid runes could look like this:
func forceASCII(s string) string {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
}
}
return string(rs)
}
// forceASCII("Hello, World!") // => "Hello, World!"
// forceASCII("Hello, 世界!") // => "Hello, !"
// forceASCII("Привет") // => ""
But what if you want special behavior if the target UTF-8 string contains any characters outside the ASCII character range of [0,127]
?
You could write a function that handles various cases by extracting a function argument which takes the invalid-ASCII rune and returns a string replacement or error.
For example (Go Playground):
func forceASCII(s string, replacer func(rune) (string, error)) (string, error) {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
} else {
replacement, err := replacer(r)
if err != nil {
return "", err
}
rs = append(rs, rune(replacement)...)
}
}
return string(rs), nil
}
func main() {
replacers := func(r rune) (string, error){
// omit invalid runes
func(_ rune) (string, error) { return "", nil },
// replace with question marks
func(_ rune) (string, error) { return "?", nil },
// abort with error */
func(r rune) (string, error) { return "", fmt.Errorf("invalid rune 0x%x", r) },
}
ss := string{"Hello, World!", "Hello, 世界!"}
for _, s := range ss {
for _, r := range replacers {
ascii, err := forceASCII(s, r)
fmt.Printf("OK: %q → %q, err=%vn", s, ascii, err)
}
}
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, 世界!" → "Hello, !", err=<nil>
// OK: "Hello, 世界!" → "Hello, ??!", err=<nil>
// OK: "Hello, 世界!" → "", err=invalid rune 0x4e16
}
add a comment |
A simple version that omits invalid runes could look like this:
func forceASCII(s string) string {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
}
}
return string(rs)
}
// forceASCII("Hello, World!") // => "Hello, World!"
// forceASCII("Hello, 世界!") // => "Hello, !"
// forceASCII("Привет") // => ""
But what if you want special behavior if the target UTF-8 string contains any characters outside the ASCII character range of [0,127]
?
You could write a function that handles various cases by extracting a function argument which takes the invalid-ASCII rune and returns a string replacement or error.
For example (Go Playground):
func forceASCII(s string, replacer func(rune) (string, error)) (string, error) {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
} else {
replacement, err := replacer(r)
if err != nil {
return "", err
}
rs = append(rs, rune(replacement)...)
}
}
return string(rs), nil
}
func main() {
replacers := func(r rune) (string, error){
// omit invalid runes
func(_ rune) (string, error) { return "", nil },
// replace with question marks
func(_ rune) (string, error) { return "?", nil },
// abort with error */
func(r rune) (string, error) { return "", fmt.Errorf("invalid rune 0x%x", r) },
}
ss := string{"Hello, World!", "Hello, 世界!"}
for _, s := range ss {
for _, r := range replacers {
ascii, err := forceASCII(s, r)
fmt.Printf("OK: %q → %q, err=%vn", s, ascii, err)
}
}
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, 世界!" → "Hello, !", err=<nil>
// OK: "Hello, 世界!" → "Hello, ??!", err=<nil>
// OK: "Hello, 世界!" → "", err=invalid rune 0x4e16
}
add a comment |
A simple version that omits invalid runes could look like this:
func forceASCII(s string) string {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
}
}
return string(rs)
}
// forceASCII("Hello, World!") // => "Hello, World!"
// forceASCII("Hello, 世界!") // => "Hello, !"
// forceASCII("Привет") // => ""
But what if you want special behavior if the target UTF-8 string contains any characters outside the ASCII character range of [0,127]
?
You could write a function that handles various cases by extracting a function argument which takes the invalid-ASCII rune and returns a string replacement or error.
For example (Go Playground):
func forceASCII(s string, replacer func(rune) (string, error)) (string, error) {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
} else {
replacement, err := replacer(r)
if err != nil {
return "", err
}
rs = append(rs, rune(replacement)...)
}
}
return string(rs), nil
}
func main() {
replacers := func(r rune) (string, error){
// omit invalid runes
func(_ rune) (string, error) { return "", nil },
// replace with question marks
func(_ rune) (string, error) { return "?", nil },
// abort with error */
func(r rune) (string, error) { return "", fmt.Errorf("invalid rune 0x%x", r) },
}
ss := string{"Hello, World!", "Hello, 世界!"}
for _, s := range ss {
for _, r := range replacers {
ascii, err := forceASCII(s, r)
fmt.Printf("OK: %q → %q, err=%vn", s, ascii, err)
}
}
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, 世界!" → "Hello, !", err=<nil>
// OK: "Hello, 世界!" → "Hello, ??!", err=<nil>
// OK: "Hello, 世界!" → "", err=invalid rune 0x4e16
}
A simple version that omits invalid runes could look like this:
func forceASCII(s string) string {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
}
}
return string(rs)
}
// forceASCII("Hello, World!") // => "Hello, World!"
// forceASCII("Hello, 世界!") // => "Hello, !"
// forceASCII("Привет") // => ""
But what if you want special behavior if the target UTF-8 string contains any characters outside the ASCII character range of [0,127]
?
You could write a function that handles various cases by extracting a function argument which takes the invalid-ASCII rune and returns a string replacement or error.
For example (Go Playground):
func forceASCII(s string, replacer func(rune) (string, error)) (string, error) {
rs := make(rune, 0, len(s))
for _, r := range s {
if r <= 127 {
rs = append(rs, r)
} else {
replacement, err := replacer(r)
if err != nil {
return "", err
}
rs = append(rs, rune(replacement)...)
}
}
return string(rs), nil
}
func main() {
replacers := func(r rune) (string, error){
// omit invalid runes
func(_ rune) (string, error) { return "", nil },
// replace with question marks
func(_ rune) (string, error) { return "?", nil },
// abort with error */
func(r rune) (string, error) { return "", fmt.Errorf("invalid rune 0x%x", r) },
}
ss := string{"Hello, World!", "Hello, 世界!"}
for _, s := range ss {
for _, r := range replacers {
ascii, err := forceASCII(s, r)
fmt.Printf("OK: %q → %q, err=%vn", s, ascii, err)
}
}
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, World!" → "Hello, World!", err=<nil>
// OK: "Hello, 世界!" → "Hello, !", err=<nil>
// OK: "Hello, 世界!" → "Hello, ??!", err=<nil>
// OK: "Hello, 世界!" → "", err=invalid rune 0x4e16
}
edited Nov 13 '18 at 22:33
answered Nov 13 '18 at 21:56
maericsmaerics
104k29201249
104k29201249
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53287230%2fascii-encoding-for-string-in-go%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
What do you expect to happen if the string contains a byte outside the range
[0,127]
?– maerics
Nov 13 '18 at 20:17