Hi there, we have a problem with unicode support on Appian cloud at two customer

Stefan Helzle

A Score Level 3

over 8 years ago

Hi there, we have a problem with unicode support on Appian cloud at two customers. A user pastes a emoji into a text field which is saved to datastore in process. The "Write to datastore" node breaks. After digging a bit I found out that MySQL is not configured to handle unicode correctly. The default charsets in MySQL are not able to save 4-byte characters. Is there anyone having a similar problem?

Discussion posts and replies are publicly visible

0 Stefan Helzle
A Score Level 3
over 8 years ago

The current MySQL config.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Stefan Helzle
A Score Level 3
over 8 years ago

The character we have a problem with is www.fileformat.info/.../index.htm
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Josh
Certified Lead Developer
over 8 years ago

Hi stefan, we have a similar issue on our project. Our workaround is to strip all unknown characters before saving into a rule input. Ex. a!save(ri!input, rule!strip(save!value)) where rule!strip is cleanwith(ri!text," abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()`~-_=+{}|:<>?[]\\;',./""")
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Stefan Helzle
A Score Level 3
over 8 years ago

Yeah, sure, we could do that. But we are in Europe with many different languages and all kinds of characters. And deleting emojis from any text is no real solution.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Josh
Certified Lead Developer
over 8 years ago

I agree it is not a real solution, just a workaround at this point. When we ran into this issue an Appian PS consultant posted about supporting emojis on Appian's home site but I don't believe anyone commented on that post.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Stefan Helzle
A Score Level 3
over 8 years ago

I can not use this character for commenting here or when using the "Ask a question" actions. So Appian itself is affected by this. And having MySQL configured correctly should be big issue. See mathiasbynens.be/.../mysql-utf8mb4
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Stefan Helzle
A Score Level 3
over 8 years ago

Sorry. Should be NO big issue.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Stefan Helzle
A Score Level 3
over 8 years ago

Josh, how long ago did you find this issue?
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 Josh
Certified Lead Developer
over 8 years ago

Back in September 2015 I believe.
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel
0 chrism967 over 7 years ago

Did you ever get a workaround that could be configured DB side? We suffer from this weekly and putting strip widths before every write to datastore is not an option as there would be literally hundreds. We manually 'fix' each one manually at the moment!
Cancel
Vote Up 0 Vote Down

Sign in to reply

Verify Answer

Cancel