Saving Emoji characters to Appian Cloud MySQL database

RE: Saving Emoji characters to Appian Cloud MySQL database

davel001150 — Wed, 06 Mar 2019 14:12:56 GMT

If it is for some reason absolutely needed, perhaps you can save the numeric code value for the character in the database and a placeholder such as _ embedded in the text to serve as where the code goes, translating it back to the emoji again when you get it back from the database.

Awful lot of work for a smiley, though.

RE: Saving Emoji characters to Appian Cloud MySQL database

Philip Snyman — Wed, 06 Mar 2019 14:01:42 GMT

In our case it was actually in the name field returned by the Google Places API. It was somewhat unexpected to say the least.

...and yes, the immediate fix was to strip out the "illegal" characters with the following expression:

joinarray(
    char(
      reject(
        fn!exact,
        code(
          ri!place.name
        ),
        000000
      )
    )
)

Where 000000 is the unicode value of the offending character.

RE: Saving Emoji characters to Appian Cloud MySQL database

davel001150 — Wed, 06 Mar 2019 13:50:47 GMT

Another thing very much worth investigating is how those emoji got into the title in the first place. It's unlikely someone would intentionally go through all the trouble of adding one to legitimate business data, but very likely that an alternate program would automatically convert a string of characters into the emoji in question.

I can't mention the number of times I've tried typing a code snippet into Skype and it's produced an Angry Face or Winking face when I actually wanted it to preserve the punctuation. Just imagine what it might produce with Javascript like this: var total = x^3>(a);) It's a particular pain point with JIRA, that won't let me punctuate the way I want without making things bold or red or whatever. Just let me add punctuation without doing weird stuff unless I tell you.

Faced with your situation, I'd want to know what program the user was using to create the string, and what key combination (such as ;)) that could produce the emoji. Since that wasn't the intent, you could use the code() function on incoming characters, and if you find one of the problem ones translate them into the original intended sequence of characters.

Alternatively, there also might be a way to turn the auto-convert-to-emoji off on the other program.

RE: Saving Emoji characters to Appian Cloud MySQL database

Josh — Wed, 06 Mar 2019 13:27:43 GMT

Depending on your strategy you could use stripwith or cleanwith

docs.appian.com/.../fnc_text_stripwith.html
docs.appian.com/.../fnc_text_cleanwith.html

RE: Saving Emoji characters to Appian Cloud MySQL database

Philip Snyman — Wed, 06 Mar 2019 10:03:21 GMT

Is there perhaps a suggested method to strip out the illegal characters, i.e. function or expression?

RE: Saving Emoji characters to Appian Cloud MySQL database

Mike Cichy — Thu, 13 Jul 2017 15:43:29 GMT

Please make sure to open a Support case to report this.

RE: Saving Emoji characters to Appian Cloud MySQL database

aloks0189 — Thu, 13 Jul 2017 06:25:45 GMT

Yes I completely agree with you mikej117

RE: Saving Emoji characters to Appian Cloud MySQL database

mikej117 — Thu, 13 Jul 2017 06:22:09 GMT

Thanks for the replies and Josh

We are on Appian cloud so updating the database collation is not possible, unless Appian Support are willing to do so.

It sounds like this is something that is better to be fixed at the product level (i.e. Appian) rather than each individual application.

RE: Saving Emoji characters to Appian Cloud MySQL database

aloks0189 — Tue, 11 Jul 2017 13:22:52 GMT

I agree with Josh unfortunately we need to alter the DB Schema as well, so i don't feel writing emoji into DB, a good choice here, specially when we are on Cloud

RE: Saving Emoji characters to Appian Cloud MySQL database

Josh — Tue, 11 Jul 2017 12:00:06 GMT

Another thing to note: emojis will break when trying to add news feed comments/posts. This might be fixed by applying the solution aloks176 suggested to the primary database, but I haven't tried that. It may be more difficult to apply those updates to the primary database if you are on cloud.

RE: Saving Emoji characters to Appian Cloud MySQL database

aloks0189 — Tue, 11 Jul 2017 06:49:17 GMT

hi mikej117 you can follow the following steps to allow MySQL Database Table to store emoji :

1) Set the database charset to utf8mb4

2) Set the charset of column to utf8mb4

Below are the commands which you would like to execute
===============================================================================================================
ALTER TABLE Tablename CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin

ALTER TABLE `comments` CHANGE `text` `text` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL;

Hope this will help You