Not able to set utf8mb4 character collation, running into encoding issues with emojis.

See original GitHub issue

Dear community, after a database migration we face serious issues in the production system, and I hope you can help me quickly!

We face issues with storing emojis and other special characters in our MySQL database. We get the following error:

Conversion from collation utf8_general_ci into utf8mb4_unicode_520_ci impossible for parameter

prisma:query INSERT INTO `app`.`show` [...]
PrismaClientUnknownRequestError: Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(Server(ServerError {
   code: 3988,
   message: "Conversion from collation utf8_general_ci into utf8mb4_0900_ai_ci impossible for parameter",
   state: "HY000" })) })
at cb (/app/node_modules/@prisma/client/runtime/index.js:38695:17) { clientVersion: '3.8.1' }
prisma:query ROLLBACK

Our MySQL cluster has utf8mb4_0900_ai_ci set as default on all levels (db, table, column) but Prisma still sends utf8 (utf8mb3) to the database. How can I configure the charset of the underlying mysql-client/connection?

I tried SQL_PRISMA_URL='mysql://user:pass@host:port/db?charset=UTF8MB4&sslmode=require&sslcert=ssl.crt&sslaccept=accept_invalid_certs' but the charset argument gets ignored. It is also not listed as an argument which is passed, so I wonder how I can specify it. Here is the docs: https://www.prisma.io/docs/concepts/database-connectors/mysql#connection-details

Querying await prisma.$queryRaw(SHOW VARIABLES LIKE 'character_set_%') returns

  • ‘character_set_client’: ‘utf8mb3’
  • ‘character_set_connection’: ‘utf8mb3’
  • ‘character_set_database’: ‘utf8mb4’
  • ‘character_set_results’: ‘utf8mb3’
  • ‘character_set_server’: ‘utf8mb4’
  • ‘character_set_system’: ‘utf8mb3’

Which shows that it is a Prisma mysql-client/connection issue – I guess?

Thanks for help! Florian

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:2
  • Comments:13 (4 by maintainers)

github_iconTop GitHub Comments

5reactions
chrisAracenaMentorSpacescommented, Aug 11, 2022

Running into this issue as well, have there been any updates?

3reactions
janpiocommented, Feb 22, 2022

No, to me this looks like a potential real bug which needs to be confirmed by anyone on our side doing a reproduction using the information your provided (which sounds elaborate enough - which is great).

There is no parameter to set the encoding/collation with Prisma right now (see https://www.prisma.io/docs/concepts/database-connectors/mysql#connection-details), as that never came up as a problem. If we can confirm this, and can not find anything else wrong, this might be the time to introduce that parameter.

Read more comments on GitHub >

github_iconTop Results From Across the Web

MySQL utf8mb4, Errors when saving Emojis - Stack Overflow
For me, it turned out that the problem lied in mysql client. ... ALTER TABLE `reactions` CHANGE `emoji` `emoji` VARCHAR(255) CHARACTER SET utf8mb4...
Read more >
Is there a MySQL character set and encoding that will allow for ...
In MySQL, collation can be set at many levels, including the client connection. That said, Unicode ( utf8 being one of the Unicode...
Read more >
MySQL driver does not support full UTF-8 (emojis ... - Drupal
Now that we require a higher MySQL version and use ASCII encoding where possible, we can just set the utf8mb4 character set by...
Read more >
Support MySQL utf8mb4 encoding | Bamboo - Jira - Atlassian
Changing the character set / collation to use utf8mb4 also means that column indexes and key lengths will need to store 4 bytes...
Read more >
Solving the Emoji/character encoding problem in Drupal 7
To fix this, you need to switch the affected MySQL table's encoding to utf8mb4 , and also switch any table columns ('fields', in...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found