Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Default to standards-compliant utf8mb4 character set #935

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rolandwalker
Copy link
Contributor

@rolandwalker rolandwalker commented Jan 7, 2021

Description

xref #915

The utf8 character set in current MySQL versions is not actually standards-compliant. The standards-compliant UTF-8 character set is spelled utf8mb4, and that should be mycli's default.

WIP because this should be researched/tested for MariaDB and Percona.
Edit: tested on MariaDB. Researched for Percona: pages such as https://www.percona.com/blog/2018/04/10/migrating-database-charsets-to-utf8mb4/ have no suggestion of incompatibility.

Checklist

  • I've added this contribution to the changelog.md.
  • I've added my name to the AUTHORS file (or it's already there).

@rolandwalker rolandwalker changed the title WIP: Default to standards-compliant utf8mb4 character set Default to standards-compliant utf8mb4 character set Jan 8, 2021
@gfrlv
Copy link
Member

gfrlv commented Jan 10, 2021

I think this default should depend on the server version: mysql < 5.5 cannot handle utf8mb4.

@rolandwalker
Copy link
Contributor Author

@pasenor excellent point.

@rolandwalker rolandwalker changed the title Default to standards-compliant utf8mb4 character set WIP Default to standards-compliant utf8mb4 character set Jan 11, 2021
@rolandwalker
Copy link
Contributor Author

Wait, how would that work? We set the charset before we make the connection.

@gfrlv
Copy link
Member

gfrlv commented Jan 11, 2021

Oops, indeed, that's messy. But we should be able call set_charset() on the PyMySQL connection object inside our SQLExecute once know the version

@dveeden
Copy link
Contributor

dveeden commented Oct 27, 2022

Note that utf8 is considered an alias for utf8mb3 and both MySQL and MariaDB are actively doing work to eventually change the alias to map to utf8mb4.

Important Change: A previous change renamed character sets having deprecated names prefixed with utf8_ to use utf8mb3_ instead. In this release, we rename the utf8_ collations as well, using the utf8mb3_ prefix; this is to make the collation names consistent with those of the character sets, not to rely any longer on the deprecated collation names, and to clarify the distinction between utf8mb3 and utf8mb4. The names using the utf8mb3_ prefix are now used exclusively for these collations in the output of SHOW statements such as SHOW CREATE TABLE, as well as in the values displayed in the columns of Information Schema tables including the COLLATIONS and COLUMNS tables. (Bug #33787300)

The utf8mb4 character set was introduced in MySQL 5.5.4, this was not a G.A. release (5.5.8 was the first G.A. release). So utf8mb4 should be used for MySQL 5.5 and newer.

@amjith
Copy link
Member

amjith commented Apr 20, 2023

@rolandwalker Is this PR still valid? Should this be merged?

@rolandwalker
Copy link
Contributor Author

@amjith yes we should do something about it. Will review.

@j-bennet j-bennet marked this pull request as draft October 16, 2023 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants