Support for Date-Time Types in Connector/J 8.0

Connector/J version 8.0.23 came out with several bug fixes related to date-time types support. They provide more flexibility for configuring time zone handling and allow migration from Connector/J 5.1 with much less effort.

Problems with migration from Connector/J 5.1 to Connector/J 8.0 were caused by the early decision that Connector/J 8.0 should always try to preserve an instant point on the time-line while Connector/J 5.1 does it optionally and, by default, preserves the original visual representation.

For example, the following code will store different results with Connector/J 5.1 and Connector/J 8.0 in case the client and server time zones are different:

Statement st = conn.createStatement();
st.executeUpdate("CREATE TABLE t1 (ts TIMESTAMP)");

PreparedStatement ps = conn.prepareStatement("INSERT INTO t1 VALUES (?)");
ps.setTimestamp(1, Timestamp.valueOf("2020-01-01 12:00:00"));
ps.executeUpdate();

If the client is running in the UTC+2 time zone and server is running in UTC+1 the internal value of the TIMESTAMP field will be “2020-01-01 11:00:00Z” with Connector/J 5.1 but “2020-01-01 10:00:00Z” with Connector/J 8.0.

Another client in the UTC+3 time zone is reading this value:

ResultSet rs = st.executeQuery("SELECT * FROM t1");
Timestamp ts = rs.getTimestamp(1);

The result will be “2020-01-01 12:00:00” with Connector/J 5.1 but “2020-01-01 13:00:00” with Connector/J 8.0.

By default, Connector/J 5.1 sends values as they are rendered locally and, on retrieval, constructs values using the client’s local time zone. Thus, the visual representation remains the same in any client time zone and on the server. But the internal UTC value of a TIMESTAMP could be different from an expected instant value.

Connector/J 8.0.22 and before converts the original value to the session time zone before sending, thus the internal UTC value of a TIMESTAMP matches the expected instant value. When retrieved, a value is constructed after converting the on-wire value from session time zone to the local one, so it still represents the same instant, but the visual representation is different in different client time zones.

Actually both approaches are valid use cases. You probably don’t care about the real internal TIMESTAMP value if you don’t do any server-side calculations with it. But you might want to store the instant point on the time-line, like for a Zoom meeting start time, in other cases. Both ways are now possible with Connector/J 8.0.23. The following connection properties define the time zone handling:

  • connectionTimeZone=LOCAL|SERVER|user-defined time zone (previously known as ‘serverTimezone’, now with additional fixed values) defines how the server’s session time zone is to be determined by Connector/J.
  • forceConnectionTimeZoneToSession=true|false controls whether the session time_zone variable is to be set to the value specified in ‘connectionTimeZone’.
  • preserveInstants=true|false turns on|off the conversion of instant values between JVM and ‘connectionTimeZone’.

The most useful configurations are:

  • connectionTimeZone=LOCAL & forceConnectionTimeZoneToSession=false – corresponds with the Connector/J 5.1 behavior with useLegacyDatetimeCode=true.
  • connectionTimeZone=LOCAL & forceConnectionTimeZoneToSession=true – the new mode which provides the most natural way for handling date-time values.
  • connectionTimeZone=SERVER & preserveInstants=true – corresponds to the previous Connector/J 8.0 behavior and Connector/J 5.1 behavior with useLegacyDatetimeCode=false.
  • connectionTimeZone=user_defined & preserveInstants=true – helps to overcome the situation when the server time zone cannot be recognized by the connector because it is set as a generic abbreviation like CET/CEST.

More details are given below, but first let’s define what exactly we mean when talking about “instant date-time classes” and “instant MySQL types”.

Supported MySQL date-time data types

The MySQL TIMESTAMP is the only data type designed to store instant points on the time-line using the implied time zone conversion. Incoming values are converted by server from the session time zone to UTC for storage, and outgoing values are converted from UTC to the session time zone. There is no way to preserve the original time zone but even if the session time zone was altered, the retrieved value still represents the same instant point on the time-line. Starting from MySQL 8.0.19, you can also specify a time zone offset when storing TIMESTAMP values (see “The DATE, DATETIME, and TIMESTAMP Types”). In such case the incoming values are converted to UTC from the specified offset instead of the session time zone. But, finally, the original offset is also lost.

The MySQL YEAR represents only the year number, DATE is missing a time part, TIME is missing the date part. So, they are definitely not instant types.

The situation with MySQL DATETIME data type is less straightforward. There is no implied time zone conversion for this type; values are stored and retrieved as they are, so this type could be taken as an analog of Java LocalDateTime. However, with the new syntax available since MySQL 8.0.19, when a value with offset is stored to a DATETIME column, it is converted to the session time zone!

New-style DATETIME values are consistent with old-style ones and the retrieved results are consistent only when client’s and session time zones are the same.

Therefore, it is more correct to say that server assumes that local date-time values coming to DATETIME always belong to the session time zone. In any case, DATETIME is not an instant type: there is no time zone information associated with a stored DATETIME value.

Users sometimes store Timestamps to DATETIME instead of TIMESTAMP because of wider ranges of represented values (‘1000-01-01 00:00:00.000000’ to ‘9999-12-31 23:59:59.999999’ against ‘1970-01-01 00:00:01.000000Z’ to ‘2038-01-19 03:14:07.999999Z’ respectively). It is possible to use DATETIME and still leverage the capability of preserving instants but with some limitations. See details in preserveInstants property description below.

Supported Java date-time classes

Connector/J 8.0.23 considers following date-time classes as “non-instant” ones even when some of them are extending the java.util.Date:

  • java.sql.Date – The time components are set to zeros.
  • java.sql.Time – The date components are set to the “zero epoch” value.
  • java.time.LocalDate – The time components and time zone are not defined.
  • java.time.LocalTime – The date components and time zone are not defined.
  • java.time.LocalDateTime – The time zone is not defined.
  • java.time.OffsetTime – The date components are not defined.

Supported “instant” date-time classes are:

  • java.util.Calendar
  • java.util.Date
  • java.sql.Timestamp
  • java.time.OffsetDateTime
  • java.time.ZonedDateTime

Preserving instants

First, let’s enumerate the time zones we are dealing with:

  • MySQL TIMESTAMP internal time zone– it’s always UTC.
  • MySQL session time zone – the per-client “on-wire” time zone, specified in the time_zone system variable. By default, the initial value of this is ‘SYSTEM’, which means, “use the value of system_time_zone”, but it may be redefined at server startup with the –default-time-zone option.
  • Client local time zone – the JVM default time zone. By default, it is equal to the client system time zone but can be changed by an application.
  • Original time zone – Objects based on java.util.Date are associated with the JVM time zone by default, but java.util.Calendar, java.time.OffsetDateTime and java.time.ZonedDateTime have an explicit time zone.

So, MySQL never stores a date-time value with its original time zone but it always assumes that this value is correctly represented in the session time zone. It means that in order to preserve the instant value, Connector/J should adjust it to the session time zone before sending to server. It can be done in different ways. Either Connector/J converts the value from the original time zone to the session time zone and then sends it, or Connector/J could set the session time zone equal to the local one so that no conversion is needed then except for java.util.Calendar, java.time.OffsetDateTime, and java.time.ZonedDateTime values, which should be converted to the local time zone.

Connector/J will never try to preserve the instant value if either the source class or the target type are not of the instant-based ones, even if Connector/J is configured to preserve instant. For example, if java.sql.Date is sent as SQL TIMESTAMP, it is rendered in the local time zone, if java.sql.Timestamp is sent as SQL DATE it is also rendered in the local time zone. But when java.sql.Timestamp is sent as SQL TIMESTAMP the value will be adjusted to the session time zone.

For example, with client time zone UTC+2, session time zone UTC+1 and connection is created with connectionTimeZone=SERVER&preserveInstants=true:

import com.mysql.cj.MysqlType;
...
ps = conn.prepareStatement("INSERT INTO t1 VALUES (?)");

ps.setObject(Date.valueOf("2020-01-01"), MysqlType.DATE);
ps.executeUpdate(); // sends "INSERT INTO t1 VALUES ('2020-01-01')"

ps.setObject(Date.valueOf("2020-01-01"), MysqlType.TIMESTAMP);
ps.executeUpdate(); // sends "INSERT INTO t1 VALUES ('2020-01-01 00:00:00')"

ps.setObject(Timestamp.valueOf("2020-01-01 00:00:00"), MysqlType.TIMESTAMP);
ps.executeUpdate(); // sends "INSERT INTO t1 VALUES ('2019-12-31 23:00:00')"

Please note, that the target type is not necessarily the real target column type. Connector/J has two implementations of PreparedStatement. ServerPreparedStatement is really prepared on the server side and returns the parameters’ metadata on prepare. But the ClientPreparedStatement is prepared on the client side and sent as a plain query on execute, thus no parameters metadata is available there. Since Connector/J could switch between these implementations internally on some conditions, they should perform identically. Thus, for the sake of consistency, the parameters metadata is not used, the decision about sent value is always based only on the default JDBC Type for the used Java class, as defined in TABLE B-4 of the JDBC specification, or on the explicitly defined target JDBC Type in a setObject() call.

MySQL DATETIME is a non-standard type but it matches perfectly with the LocalDateTime class by nature, thus the default target type for LocalDateTime is MysqlType.DATETIME instead of TIMESTAMP, as per the JDBC specification.

Time zone configuration properties

  • connectionTimeZone=LOCAL|SERVER|user-defined time zone. The ‘serverTimezone’ property was renamed to ‘connectionTimeZone’ in order to highlight that it does not necessary match the real server or session time zone. It just informs the connector which time zone should be used in cases where the instant value must be converted between the JVM and the on-wire time zones. If it is set to “LOCAL” (which is the default value, if the property is not specified) the driver assumes that the connection time zone is the same as the JVM default time zone. If set to “SERVER” then the driver attempts to detect the session time zone from the values configured on the MySQL server session variables “time_zone” or “system_time_zone”. An explicit value must be a geographic time zone name or a time zone offset from Greenwich/UTC, using a syntax java.time.ZoneId is able to parse.

This option itself does not set MySQL server session variable ‘time_zone’ to the given value. To do that the ‘forceConnectionTimeZoneToSession’ connection option must be set to “true”.

Please note that setting a value to ‘connectionTimeZone’ in conjunction with ‘forceConnectionTimeZoneToSession=false’ and ‘preserveInstants=false’ has no effects since, in that case, connectionTimeZone is used neither to change the session time zone nor to convert the time zone.

  • forceConnectionTimeZoneToSession=true|false. If it is set to “false” (which is the default value) the session time zone is left unchanged on the server. When this property is set to “true”, the driver sets the time zone value determined by ‘connectionTimeZone’ connection property to the current server session ‘time_zone’ variable.

Please be aware that altering the session time zone also affects the result of MySQL functions such as ‘NOW()’, ‘CURTIME()’ or ‘CURDATE()’.

This option has no effect if used in conjunction with ‘connectionTimeZone=SERVER’ since, in this case, you are requesting the session time zone to be set to the value it already has.

  • preserveInstants=true|false. With ‘preserveInstants=false’ Connector/J 8.0.23 always uses the JVM default time zone for rendering the values it sends to the server and for constructing the Java objects from the fetched data. It matches the default Connector/J 5.1 behavior. If ‘preserveInstants=true’ (which is the default value), Connector/J does its best to preserve the instant point on the time-line for Java instant-based objects such as java.sql.Timestamp or java.time.OffsetDateTime, instead of preserving the time’s original visual form. When storing, the conversion is performed only if the target SQLType, either the explicit one or the default one, is TIMESTAMP. When retrieving, the conversion is performed only if the source column has the TIMESTAMP, DATETIME or character type and the target class is an instant-based one, like java.sql.Timestamp or java.time.OffsetDateTime.

This option has no effect if used in conjunction with ‘connectionTimeZone=LOCAL’ since, in this case, the source and target time zones are the same.

Conversion between original time zone and session time zone has a risk of altering the value in case they have incompatible DST rules.

Time zone configurations

  • connectionTimeZone=LOCAL & forceConnectionTimeZoneToSession=false. No time zone conversion is done by the driver because it assumes that the session time zone equals to the local one. The actual session time zone is left untouched and it is the same for all clients. That’s the way for keeping the same visual date-time value representation. Instants may be kept correctly only when all clients and the server are actually residing in the same time zone.

This configuration corresponds to the default Connector/J 5.1 behavior (with useLegacyDatetimeCode=true).

  • connectionTimeZone=LOCAL & forceConnectionTimeZoneToSession=true. This new mode provides the most natural way for handling date-time values. Connector/J sets the session time zone equal to the local one and always uses the local time zone internally. Server converts a TIMESTAMP to its internal UTC value directly from the client time zone eliminating possible problems with DST. It’s also safe to use both the old syntax and the extended syntax with zone offset for setting DATETIMEs–they are finally adjusted to the same session time zone.

Since different clients in this mode might change their individual session into different time zones, this mode does not allow the same visual date-time value representation to be kept in a TIMESTAMP column; store the value to a DATETIME column instead.

And, again, altering the session time zone also affects the result of MySQL functions such as ‘NOW()’, ‘CURTIME()’ or ‘CURDATE()’.

  • connectionTimeZone=SERVER & preserveInstants=true. The session time zone is left untouched and it is the same for all clients. The driver converts instant values to the session time zone when they are sent as a TIMESTAMP. When retrieving, the driver converts values from the session time zone to the local one if the source column is in the TIMESTAMP, DATETIME or character type and the target class is an instant-based one.

This configuration corresponds to the previous Connector/J 8.0 behavior and Connector/J 5.1 behavior with useLegacyDatetimeCode=false.

  • connectionTimeZone=user_defined & preserveInstants=true. Without changing the session time zone on the server, this setup helps to overcome the situation in which the server time zone cannot be recognized by the connector because it is set with an unrecognizable, generic abbreviation like CET or CEST. This configuration gives Connector/J the proper time zone identification for the session, to which the timestamps should be converted.

OffsetDateTime and ZonedDateTime support

The original time zone of java.time.OffsetDateTime and java.time.ZonedDateTime is never preserved when they are stored to the date-time column. The driver first converts the date-time values into the local time zone and then, if it is preserving instants, it converts them to the session time zone.

When the driver reads instants from the server into these objects and it is configured to preserve instants, the result will be constructed with the session time zone. Otherwise, result will be constructed with the local time zone.

For example, with client time zone UTC+2, session time zone UTC+1 and the connection created with connectionTimeZone=SERVER&preserveInstants=true:

import com.mysql.cj.MysqlType;
...
ps = conn.prepareStatement("INSERT INTO t1 VALUES (?)");

ps.setObject(OffsetDateTime.parse("2020-01-01T13:00:00+03:00"));
ps.executeUpdate();
// sends "INSERT INTO t1 VALUES ('2020-01-01 11:00:00')"

ResultSet rs = st.executeQuery("SELECT * FROM t1");
String odt = rs.getObject(1, OffsetDateTime.class).toString();
// returns "2020-01-01T11:00:00+01:00"

Fractional seconds support

New connection property sendFractionalSecondsForTime=true|false tells the driver to send or ignore the fractional seconds contained in java.sql.Time.

This option overrides the inconsistency between MySQL TIME and JDBC java.sql.Time. While MySQL TIME may store fractional seconds the java.sql.Time is defined as a time value without a fractional part. On the other hand, java.sql.Time is a wrapper around the java.util.Date, thus internally it also may contain fractional seconds. So, when sending the java.sql.Time with ‘sendFractionalSecondsForTime=true’, its value is rendered with fractional seconds; with ‘sendFractionalSecondsForTime=false’, the value is rendered without fractional seconds.

This option is not applied to getters–values from MySQL TIME are always constructed with a fractional part if provided by server.

Connection property sendFractionalSeconds=true|false is redefined as a global switch. If set to “false”, fractional seconds are rounded or truncated not only for TIMESTAMP but also for any other date-time type.

Additional Information and Resources

Deprecating End-of-life runtime engines on Connector/Node.js

Starting with the 8.0.23 release, Connector/Node.js will be deprecating support for End-of-life Node.js engine versions. Support for these versions will eventually be removed on subsequent releases which will focus on compatibility with the available LTS versions at the date of each release.

Continue reading

MySQL Connector/ODBC 8.0.23 has been released

Dear MySQL users,

MySQL Connector/ODBC 8.0.23 is a new version in the MySQL Connector/ODBC
8.0 series, the ODBC driver for the MySQL Server.

The available downloads include both a Unicode driver and an ANSI driver
based on the same modern codebase. Please select the driver type you
need based on the type of your application – Unicode or ANSI.
Server-side prepared statements are enabled by default. It is suitable
for use with the latest MySQL server version 8.0.

This release of the MySQL ODBC driver is conforming to the ODBC 3.8
specification. It contains implementations of key 3.8 features,
including self-identification as a ODBC 3.8 driver, streaming of out (for
binary types only), and support of the SQL_ATTR_RESET_CONNECTION
connection attribute (for the Unicode driver only).

The release is now available in source and binary form for a number of
platforms from our download pages at

https://dev.mysql.com/downloads/connector/odbc/

For information on installing, please see the documentation at

https://dev.mysql.com/doc/connector-odbc/en/connector-odbc-installation.html

Enjoy and thanks for your support!

Changes in MySQL Connector/ODBC 8.0.23 (2021-01-18, General Availability)

Bugs Fixed

* Using the commercial glib package
(authentication_ldap_sasl) with auth using GSSAPI
(authentication_ldap_sasl_auth_method_name=’GSSAPI’) did
not function. Added the missing sasl2 modules package.
(Bug #32175842)

* On Debian based systems, the post-installation script
uses dpkg-architecture ODBC drivers path, so the
associated dpkg-dev package was added as a runtime
dependency. (Bug #32157740)

* Microsoft Access halted when opening a linked table with
only a JSON column. The JSON column type is now
supported, and JSON data is only editable if another
column is used as a primary key. Because JSON is a long
data type with the maximum length of 4GB, it can’t be
used as a primary key by Microsoft Access and therefore
tables having only JSON columns are only available in
read-only mode. (Bug #32114901)

* Removed the mysql-client-plugins dependency. It remains
required for connections using commercial MySQL server
accounts with LDAP authentication, so must be manually
installed for that situation. The mysql-client-plugins
package has conflicts with MySQL server versions before
8.0.21, so earlier versions (such as MySQL 5.7) require
an 8.0 server upgrade to use it. (Bug #31875490)

* On macOS, Connector/ODBC would not report an error if
SQL_ATTR_PARAMSET_SIZE was set but not supported; instead
the setting was ignored. (Bug #29862441, Bug #95608)

On Behalf of Oracle/MySQL Release Engineering Team,
Hery Ramilison

MySQL Connector/Python 8.0.23 has been released

Dear MySQL users,

MySQL Connector/Python 8.0.23 is the latest GA release version of the MySQL Connector Python 8.0 series. The X DevAPI enables application developers to write code that combines the strengths of the relational and document models using a modern, NoSQL-like syntax that does not assume previous experience writing traditional SQL.

To learn more about how to write applications using the X DevAPI, see

http://dev.mysql.com/doc/x-devapi-userguide/en/

For more information about how the X DevAPI is implemented in MySQL Connector/Python, and its usage, see

http://dev.mysql.com/doc/dev/connector-python

Please note that the X DevAPI requires at least MySQL Server version 8.0 or higher with the X Plugin enabled. For general documentation about how

http://dev.mysql.com/doc/refman/8.0/en/document-store.html

To download MySQL Connector/Python 8.0.23, see the “General Availability (GA) Releases” tab at

http://dev.mysql.com/downloads/connector/python/

Enjoy!

Changes in MySQL Connector/Python 8.0.23 (2021-01-18, General Availability)

Functionality Added or Changed

  • Deprecated Python 2.7 support; a warning was added to describe Python 2.7 support being removed in Connector/Python 8.0.24.

    Previously, Connector/Python added client support for the MySQL Enterprise Edition SASL LDAP authentication plugin with SCRAM-SHA-1 as an authentication method. Connector/Python now also supports SCRAM-SHA-256 as an alternative authentication method for classic MySQL protocol connections. SASL-based LDAP authentication does not apply to clients running macOS.

  • Added SASL authentication protocol support using the GSSAPI (Kerberos) authentication method for the pure python implementation (this is not available for the c-ext version of Connector/Python).

    This functionality requires the GSSAPI pypi module, which provides both low-level and high-level wrappers around the GSSAPI C libraries. The GSSAPI pypi module requires MIT kerberos installed opn the system to function and request tickets to authenticate Connector/Python with the MySQL server when the user is IDENTIFIED WITH authentication_ldap_sasl and the authentication_ldap_sasl plugin is configured to use the GSSAPI mechanism.

    This also adds a new krb_service_principal option, which must be a string in the form “primary/instance@realm” such as “ldap/ldapauth@MYSQL.COM” where “@realm” is optional. The “@realm” defaults to the default realm, as configured in the krb5.conf file.

Bugs Fixed

  • Fixed the AttributeError raised when getting the connection ID from a closed CMySQLConnection. (Bug #31882419, Bug #100825)

  • Fixed support for named tuple (an invalid exception was generated) and dictionary cursors (incorrect type of result was returned). (Bug #29195610)

  • Fixed cursor.fetchone() and cursor.fetchmany() to comply with PEP 249, which specifies that an exception must be raised if the previous call to cursor.execute*() does not produce any result set or no call was issued yet. (Bug #26834307, Bug #87815)

  • Fixed the microsecond conversion from MySQL datetime to Python datetime when using fractional values with the C extension enabled. For example, 2016-10-20 15:40:23.8 from MySQL resulted in datetime.datetime(2016, 10, 20, 15, 40, 23, 8) instead of datetime.datetime(2016, 10, 20, 15, 40, 23, 800000). Thanks to Vilnis Termanis for the patch. (Bug #24938411, Bug #83479)

MySQL Connector/Node.js 8.0.23 has been released


Dear MySQL users,

MySQL Connector/Node.js is a new Node.js driver for use with the X
DevAPI. This release, v8.0.23, is a maintenance release of the
MySQL Connector/Node.js 8.0 series.

The X DevAPI enables application developers to write code that combines
the strengths of the relational and document models using a modern,
NoSQL-like syntax that does not assume previous experience writing
traditional SQL.

MySQL Connector/Node.js can be downloaded through npm (see
  https://www.npmjs.com/package/@mysql/xdevapi for details) or from
  https://dev.mysql.com/downloads/connector/nodejs/.

To learn more about how to write applications using the X DevAPI, see
  http://dev.mysql.com/doc/x-devapi-userguide/en/.
For more information about how the X DevAPI is implemented in MySQL
Connector/Node.js, and its usage, see
  http://dev.mysql.com/doc/dev/connector-nodejs/.

Please note that the X DevAPI requires at least MySQL Server version
8.0 or higher with the X Plugin enabled. For general documentation
about how to get started using MySQL as a document store, see
  http://dev.mysql.com/doc/refman/8.0/en/document-store.html.

Changes in MySQL Connector/Node.js 8.0.23 (2021-01-18, General Availability)

* Functionality Added or Changed
* Bugs Fixed

Functionality Added or Changed

* Added custom debug functionality using the NODE_DEBUG
environment variable to log and inspect low-level details
for the application. Connector/Node.js now supports this
feature and uses it, in particular, for logging
information about the protocol messages (inbound and
outbound) that are exchanged with the MySQL server.
Messages sent by the client are available under the
protocol:outbound scope, whereas messages sent by the
server are available under the protocol:inbound scope.
For example, the following writes a textual protobuf
representation of every Mysqlx.Crud.Find and
Mysqlx.Resultset.Row message to stderr: shell> NODE_DEBUG=’protocol:outbound:Mysqlx.Crud.Find,protocol:inbound :Mysqlx.Resultset.Row’ node app.js

Node 10 and later supports wildcard pattern matching,
such as NODE_DEBUG=’protocol:inbound:*’ to filter out
inbound messages and NODE_DEBUG=’*’ to show all logs.
(Bug #31729043)

* Added a deprecation warning to the Connector/Node.js
installation process that future Connector/Node.js
versions do not guarantee compatibility with End-of-life
Node.js versions. If you are planning to upgrade, make
sure you are using Node.js 12.0.0 or higher.

Bugs Fixed

* Stored values in a DOUBLE column were truncated when
encoded and sent to the MySQL server. For example, a
value such as 1.000001 would become 1. They are now
encoded properly as double-precision floating numbers to
support the full range of 64-bit floating point precision
in MySQL columns with the DOUBLE data type (having in
mind possible rounding an approximations performed by the
database engine). (Bug #31734504)

* Improved consistency for method argument error handling
when an argument is not set or uses JavaScript’s
“undefined”. (Bug #31709879)

* Refactored code to disable all debugging utilities when
debug mode is off. (Bug #31584269)

On Behalf of MySQL Release Engineering Team,
Surabhi Bhat

MySQL Connector/NET 8.0.23 has been released

Dear MySQL users,

MySQL Connector/NET 8.0.23 is the latest General Availability release
of the MySQL Connector/NET 8.0 series. This version supports .NET 5.0
and the X DevAPI, which enables application developers to write code
that combines the strengths of the relational and document models
using a modern, NoSQL-like syntax that does not assume previous
experience writing traditional SQL.

To learn more about how to write applications using the X DevAPI, see

  http://dev.mysql.com/doc/x-devapi-userguide/en/

For more information about how the X DevAPI is implemented in
Connector/NET, see

  http://dev.mysql.com/doc/dev/connector-net

NuGet packages provide functionality at a project level. To get the
full set of features available in Connector/NET such as availability
in the GAC, integration with Visual Studio’s Entity Framework Designer
and integration with MySQL for Visual Studio, installation through the
MySQL Installer or the stand-alone MSI is required.

Please note that the X DevAPI requires at least MySQL Server version
8.0 or higher with the X Plugin enabled. For general documentation
about how to get started using MySQL as a document store, see

  http://dev.mysql.com/doc/refman/8.0/en/document-store.html

To download MySQL Connector/NET 8.0.23, see

  http://dev.mysql.com/downloads/connector/net/

Installation instructions can be found at

  https://dev.mysql.com/doc/connector-net/en/connector-net-installation.html

Enjoy!

Changes in MySQL Connector/NET 8.0.23 (2021-01-18, General
Availability)


Functionality Added or Changed

     * Connector/NET updates to SSH ciphers and algorithms are:

          + Encryptions: aes192-cbc, aes256-cbc (deprecated);
            3des-cbc, blowfish-cbc, twofish-cbc, twofish128-cbc,
            twofish192-cbc, twofish256-cbc (invalid)

          + Host Key Algorithms: ssh-rsa (deprecated); sh-dss
            (invalid)

          + Key Exchange Algorithms:
            diffie-hellman-group-exchange-sha1 (invalid)

          + Keyed Hash Message Authentication Codes:
            hmac-ripemd160,hmac-ripemd160@openssh.com,
            hmac-sha1-96 (invalid)

       For a complete list, see SSH Ciphers in Connector/NET
       https://dev.mysql.com/doc/connector-net/en/connector-net-ssh-ciphers.html
       (Bug #31917057)

     * The IgnorePrepare connection-string option is deprecated
       in this release and the default value, which was true, is
       now false. When set to true, the option instructs
       Connector/NET to ignore any calls to
       MySqlCommand.Prepare() made with the classic MySQL
       protocol. The option is scheduled for removal in the next
       release. (Bug #31872906)

     * ASP.NET applications using the MySQL provider model
       (MySql.Web) can now target .NET Framework 4.8.
       (Bug #31799902)

     * The following synonyms for the Server connection string
       option were deprecated in Connector/NET 8.0.22 and
       removed in 8.0.23: address, addr, and network address.
       (Bug #31248601)

     * Previously, Connector/NET added client support for the
       MySQL Enterprise Edition SASL LDAP authentication plugin
       with SCRAM-SHA-1 as an authentication method.
       Connector/NET now also supports SCRAM-SHA-256 as an
       alternative authentication method for classic MySQL
       protocol connections. SCRAM-SHA-256 is similar to
       SCRAM-SHA-1 but is more secure. SASL-based LDAP
       authentication does not apply to clients running macOS.

     * With the availability of Entity Framework Core 5.0 in
       addition to Entity Framework Core 3.1, Connector/NET now
       provides two distinct EF Core packages. The split enables
       the connector to support both feature sets as they
       diverge. Initially, Connector/NET supports a partial EF
       Core 5.0 feature set (equivalent to EF Core 3.1) in this
       release. For more information, see Entity Framework Core
       Support
       https://dev.mysql.com/doc/connector-net/en/connector-net-entityframework-core.html
       Breaking change: all MySql.Data.EntityFrameworkCore.xxx
       namespaces are renamed to MySql.EntityFrameworkCore.xxx.
       New EF Core package naming for Connector/NET 8.0.23:

          + 5.0.0-m8.0.23
          + 3.1.10-m8.0.23

Bugs Fixed

     * If a prepared statement had no parameters, Connector/NET
       included in the COM_STMT_EXECUTE packet structure a byte
       corresponding to new-params-bound-flag instead of sending
       the byte only when the number of parameters was greater
       than zero. (Bug #32208427)

     * Incomplete validation limited the expected range of
       values that a stored procedure with a parameter of type
       Boolean could assign using the MySqlParameter.MySqlDbType
       property. (Bug #32066024, Bug #101302)

     * Stronger validation was applied to information contained
       in the certificate store for connections made using
       SslMode. (Bug #31954655)

     * A connection timeout was added to prevent the
       MySqlConnection.Open method from waiting indefinitely for
       a response after MySQL Router restarted unexpectedly.
       (Bug #31945397, Bug #100692)

     * A cast made with the wrong data type during a valid EF
       Core operation returned an exception.
       (Bug #31860492, Bug #100773)

     * Connector/NET used the value of -1 internally to ensure
       that a parameter without an index was added to the end of
       the parameter list. However, if an index with an actual
       value of -1 was passed in, the collection was interpreted
       as having no index and the argument did not generate an
       out-of-range exception. (Bug #31754599, Bug #100522)

     * Without validation, an underlying 64-bit enumeration
       value passed in as a MySQL command parameter defaulted to
       type Int32 and produced an overflow exception.
       (Bug #25467610, Bug #84701)

Enjoy and thanks for the support!

On behalf of the MySQL Release Team,
Nawaz Nazeer Ahamed

MySQL Connector/C++ 8.0.23 has been released

Dear MySQL users,

MySQL Connector/C++ 8.0.23 is a new release version of the MySQL
Connector/C++ 8.0 series.

Connector/C++ 8.0 can be used to access MySQL implementing Document
Store or in a traditional way, using SQL queries. It allows writing
both C++ and plain C applications using X DevAPI and X DevAPI for C.
It also supports the legacy API of Connector/C++ 1.1 based on JDBC4.

To learn more about how to write applications using X DevAPI, see
“X DevAPI User Guide” at

https://dev.mysql.com/doc/x-devapi-userguide/en/

See also “X DevAPI Reference” at

https://dev.mysql.com/doc/dev/connector-cpp/devapi_ref.html

and “X DevAPI for C Reference” at

https://dev.mysql.com/doc/dev/connector-cpp/xapi_ref.html

For generic information on using Connector/C++ 8.0, see

https://dev.mysql.com/doc/dev/connector-cpp/

For general documentation about how to get started using MySQL
as a document store, see

http://dev.mysql.com/doc/refman/8.0/en/document-store.html

To download MySQL Connector/C++ 8.0.23, see the “General Availability (GA)
Releases” tab at

https://dev.mysql.com/downloads/connector/cpp/

Changes in MySQL Connector/C++ 8.0.23 (2021-01-18, General Availability)

     * Legacy (JDBC API) Notes

     * Functionality Added or Changed

     * Bugs Fixed

Legacy (JDBC API) Notes

    * Previously, to build or run applications that use the
       legacy JDBC API, it was necessary to have Boost
       installed. Boost is no longer required for such
       applications. The API has not changed, so no code changes
       are required to build applications. However, in
       consequence of this change, the ABI version has increased
       from 7 to 9. To run applications, a version of
       Connector/C++ built with the same ABI must be installed:

          + Applications built using the new ABI require a
            version of Connector/C++ also built using the new
            ABI.

          + Applications built using the old ABI require a
            version of Connector/C++ also built using the old
            ABI.
       To build the legacy connector itself from source, it is
       still necessary to have Boost installed.

Functionality Added or Changed


     * All calls that allow a column name, such as findColumn(),
       getString(), and getInt(), are now case-sensitive. (Bug
       #30126457, Bug #96398)

     * The developer documentation was improved regarding how to
       decode the bytes received by mysqlx_get_bytes(). Thanks
       to Daniël van Eeden for pointing at the missing
       documentation. (Bug #29115299, Bug #93641)

     * Thanks to Daniël van Eeden, who contributed various
       corrections to the developer documentation. (Bug
       #29038157, Bug #93549)

     * A dependency on the mysql-client-plugins package was
       removed. This package now is required only on hosts where
       Connector/C++ applications make connections using
       commercial MySQL server accounts with LDAP
       authentication. In that case, additional libraries must
       also be installed: cyrus-sasl-scram for installations
       that use RPM packages and libsasl2-modules-gssapi-mit for
       installations that use Debian packages. These SASL
       packages provide the support required to use the
       SCRAM-SHA-256 and GSSAPI/Kerberos authentication methods
       for LDAP.
       If Connector/C++ applications do not use LDAP
       authentication, no additional packages are required.

Bugs Fixed


     * Connector/C++ 8.0 RPM packages could not be installed on
       a system where MySQL 5.7 RPM packages were installed.
       (Bug #32142148)

     * Establishing a connection using a ConnectOptionsMap
       object could fail due to differences in std::string
       implementations. (Bug #32039929)

     * Commercial Connector/C++ RPM packages were missing
       provides information. (Bug #31775733)

On Behalf of MySQL Release Engineering Team,
Tvarita Jain

MySQL Connector/J 8.0.23 has been released

Dear MySQL users,

MySQL Connector/J 8.0.23 is the latest General Availability release of
the MySQL Connector/J 8.0 series. It is suitable for use with MySQL
Server versions 8.0, 5.7, and 5.6. It supports the Java Database
Connectivity (JDBC) 4.2 API, and implements the X DevAPI.

This release includes the following new features and changes, also
described in more detail on

https://dev.mysql.com/doc/relnotes/connector-j/8.0/en/news-8-0-23.html

As always, we recommend that you check the “CHANGES” file in the
download archive to be aware of changes in behavior that might affect
your application.

To download MySQL Connector/J 8.0.23 GA, see the “General Availability
(GA) Releases” tab at http://dev.mysql.com/downloads/connector/j/

Enjoy!

Changes in MySQL Connector/J 8.0.23 (2021-01-18, General Availability)

Deprecation and Removal Notes

* As an implementation of the MySQL Terminology Updates (https://mysqlhighavailability.com/mysql-terminology-updates/),
connection properties and public method names have
been adjusted in the following manners:

  • Changing “master” to “source”: For example, the
    connection property queriesBeforeRetryMaster becomes
    queriesBeforeRetrySource, and the method
    isMasterConnection() becomes isSourceConnection()

  • Changing “slave” to “replica”: For example, the
    connection property allowSlavesDownConnections
    becomes allowReplicaDownConnections, and the method
    getSlaveHosts() becomes getReplicaHosts()

  • Changing “blacklist” to “blocklist”: For example,
    the connection property loadBalanceBlacklistTimeout
    becomes loadBalanceBlocklistTimeout.

Old names have been deprecated—though they are still
usable for now, they are to be removed eventually in
future releases; users are therefore encouraged to switch
to the new names.

See the MySQL Connector/J 8.0 Developer Guide
(https://dev.mysql.com/doc/connector-j/8.0/en/), the
Connector/J API documentation (generated by Javadoc), and
the MySQL Connector/J X DevAPI Reference for information
on any new property and method names.

Functionality Added or Changed

* While a java.sql.TIME instance, according to the JDBC
specification, is not supposed to contain fractional
seconds by design, because java.sql.TIME is a wrapper
around java.util.Date, it is possible to store fractional
seconds in a java.sql.TIME instance. However, when
Connector/J inserted a java.sql.TIME into the server as a
MySQL TIME value, the fractional seconds were always
truncated. To allow the fractional seconds to be sent to
the server, a new connection property,
sendFractionalSecondsForTime, has been introduced: when
the property is true (which is the default value), the
fractional seconds for java.sql.TIME are sent to the
server; otherwise, the fractional seconds are truncated.
Also, the connection property sendFractionalSeconds has
been changed into a global control for the sending of
fractional seconds for ALL date-time types. As a result,
if sendFractionalSeconds=false, fractional seconds are
not sent irrespective of the value of
sendFractionalSecondsForTime.
(Bug #20959249, Bug #76775)

* Connector/J now supports the following authentication
methods for LDAP Pluggable Authentication
(https://dev.mysql.com/doc/refman/8.0/en/ldap-pluggable-authentication.html)
with the MySQL Enterprise Server:

  • The GSSAPI/Kerberos Authentication Method:
    (https://dev.mysql.com/doc/refman/8.0/en/ldap-pluggable-authentication.html#ldap-pluggable-authentication-gssapi)
    A new connection property,
    ldapServerHostname, has been introduced for
    specifying the LDAP service host principal as
    configured in the Kerberos key distribution centre
    (KDC). See the description for ldapServerHostname in
    the MySQL Connector/J 8.0 Developer Guide
    (https://dev.mysql.com/doc/connector-j/8.0/en/) for details.

  • The SCRAM-SHA-256 method.

Bugs Fixed

* Storing a java.time.LocalDateTime object onto the server
as a TIMESTAMP value using a batched PreparedStatement
failed with the complaint that java.time.LocalDateTime
could not be cast to java.sql.Timestamp. With this fix,
the casting works again.
(Bug #32099505, Bug #101413)

* Using the setObject() method to set a
ByteArrayInputStream instance for a PreparedStatement
resulted in a SQLException. (Bug #32046007, Bug #101242)

* The returned value for a TIMESTAMP was incorrect when a
temporal interval expression
(https://dev.mysql.com/doc/refman/8.0/en/expressions.html#temporal-intervals)
was used in the SQL statement for
the query. (Bug #31074051, Bug #99013)

* After upgrading from Connector/J 5.1 to 8.0, the results
of saving and then retrieving DATETIME and TIMESTAMP
values became different sometimes. It was because while
Connector/J 5.1 does not preserve a time instant by
default, Connector/J 8.0.22 and earlier tried to so by
converting a timestamp to the server’s session time zone
before sending its value to the server. In this release,
new mechanisms for controlling timezone conversion has
been introduced—see Preserving Time Instants
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-time-instants.html)
for details. Under this new
mechanism, the default behavior of Connector/J 5.1 in
this respect is preserved by setting the connection
property preserveInstants=false. (Bug #30962953, Bug
#98695, Bug #30573281, Bug #95644)

* Conversion of a MySQL DATETIME or TIMESTAMP value to a
Java OffsetDateTime using the getObject(i,
OffsetDateTime.class) method failed with a “Conversion
not supported for type …” error. It was because the
OffsetDateTime.parse() method on DATETIME and TIMESTAMP
values yielded an unexpected string format. With this
patch, conversions between OffsetDateTime and the DATE,
TIME, DATETIME, TIMESTAMP, and YEAR data types are now
possible, and an instant point on the timeline is
preserved as such during a conversion, when
possible—see Preserving Time Instants
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-time-instants.html)
for details. (Bug #29402209, Bug #94457)

* When the server’s session time zone setting was not
understandable by Connector/J (for example, it was set to
CEST), a connection could not be established with the
server unless Connector/J specified the correct IANA time
zone name in the serverTimezone connection property. This
happened even if there was actually no need to use any
date-time functionality in Connector/J. The issue was
fixed by the new connection properties for Connector/J
that control date-time handling—see Preserving Time Instants
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-time-instants.html)
for details. The following now
happens with respect to the above-mentioned situation:

  • If the new connection property connectionTimeZone is
    set to LOCAL or a specified time zone, the time_zone
    variable on the server is no longer checked

  • If connectionTimeZone=SERVER, the check for the
    time_zone variable is delayed until date-time driver
    functionality is first invoked, so that an
    unrecognizable server time zone does not prevent
    connection to be established. However, when
    date-time functionality is invoked and the value of
    time_zone cannot be recognized by Connector/J, an
    exception is thrown.

(Bug #21789378)

Enjoy and thanks for the support!

On behalf of the MySQL Release Team,
Balasubramanian Kandasamy

MySQL Shell 8.0.23 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.23 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.23.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster and InnoDB
ReplicaSet, providing integrated solutions for high availability and scalability
using InnoDB based MySQL databases, without requiring advanced MySQL
expertise.  For more information about how to configure and work with
MySQL InnoDB cluster and MySQL InnoDB ReplicaSet see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.  For more
information about how to use MySQL Shell and the MySQL Document Store
support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.23, see the
“General Availability (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

Changes in MySQL Shell 8.0.23 (2021-01-18, General Availability)

     * AdminAPI Added or Changed Functionality

     * AdminAPI Bugs Fixed

     * Functionality Added or Changed

     * Bugs Fixed

AdminAPI Added or Changed Functionality


     * The output of the status() operation has been extended to
       provide more information relevant to diagnosing errors.
       The following information is available for InnoDB
       Clusters and InnoDB ReplicaSets:

          + the memberState field shows the actual status of the
            instance as queried locally, which can be one of
            offline, error, recovering, or online.

          + a recovery.recoveryChannel field shows instances
            performing incremental recovery or in which the
            recovery channel status is not off

          + a new instanceErrors field exists for each instance,
            displaying any diagnostic information that can be
            detected for it

          + when the extended option is set to greater than 0,
            the output includes an applierChannel field, with
            replication information if the instance is either
            online and the applier channel status is not on, or
            the status is not recovering or online and the
            applier channel status is not off
       For more information, see Checking a cluster’s Status
       with Cluster.status()
       (https://dev.mysql.com/doc/mysql-shell/8.0/en/monitoring-
       innodb-cluster.html#check-innodb-cluster-status).

     * InnoDB Cluster and InnoDB ReplicaSet now support and
       enable parallel replication appliers, sometimes referred
       to as a multi-threaded replica. With the advances in
       MySQL such as binary log transaction dependency tracking
       and XXHASH64 based GTID set extraction, using multiple
       replica applier threads improves the throughput of both
       the replication applier and incremental recovery.
       This has resulted in the following changes:

          + the requirements for instances running 8.0.23 and
            later now also include:
               o binlog_transaction_dependency_tracking=WRITESET
               o slave_preserve_commit_order=ON
               o slave_parallel_type=LOGICAL_CLOCK
               o transaction_write_set_extraction=XXHASH64
            this means that new instances running 8.0.23 have
            these options configured by dba.configureInstance()
            and dba.configureReplicaSetInstance(). Attempting to
            add an instance running version 8.0.23 or later
            which does not have these variables configured
            results in an error. When you upgrade a cluster or
            replica set that has been running a version of MySQL
            server and MySQL Shell earlier than 8.0.23, the
            parallel replication applier is not enabled on the
            instances. This means you are not taking advantage
            of this feature, and you should reconfigure your
            instances to use the parallel replication applier.
            For more information, see Configuring the Parallel
            Replication Applier
            (https://dev.mysql.com/doc/mysql-shell/8.0/en/config
            uring-innodb-cluster.html#configuring-parallel-appli
            er).

          + dba.checkInstanceConfiguration() validates if
            parallel replication appliers are enabled or not.

          + the new applierWorkerThreads option configures the
            number of replication applier threads the instance
            uses for replication, and defaults to 4 threads. Use
            this option with the dba.configureInstance() and
            dba.configureReplicaSetInstance(). You can change
            this option while the instance is online, but the
            change is only made after the instance is restarted.

          + the output of the .status(extended=1) and options()
            operations now includes information about the
            configuration of parallel appliers.

AdminAPI Bugs Fixed


     * The fix for Bug#29305551 extended the
       dba.checkInstanceConfiguration() operation to include a
       check to verify if asynchronous replication is configured
       and running on the target instance, and print a warning
       when that is the case. This check is also used by the
       Cluster.addInstance() and Cluster.rejoinInstance()
       operations to terminate them with an error when such a
       scenario is detected, and is also used by the
       dba.rebootClusterFromCompleteOutage() operation whenever
       there are instances to be rejoined to the cluster.
       However, the dba.createCluster() operation was
       erroneously skipping the check, and the
       dba.rebootClusterFromCompleteOutage() operation was
       skipping the check on the instance being used to
       bootstrap the cluster. The fix ensures that the check is
       also performed whenever creating or rebooting a cluster
       from complete outage. Additionally, it adds support to
       override the check for the dba.createCluster() operation
       by making use of the force option, and it improves the
       error messages. (Bug #32197222)

     * The fix for Bug#29305551 extended
       dba.checkInstanceConfiguration() to verify if
       asynchronous replication is configured and running on the
       target instance and print a warning if that was the case.
       However, the check missed verifying if the replication
       channel was configured but not running. This fix ensures
       the verification also considers replication channels
       which are configured but are not actively running.
       Additionally, an erroneous message which suggested the
       possibility of using STOP REPLICA to override this check
       has been removed and replaced with an informative message
       which explains that unmanaged replication channels are
       not supported and the possible dangers of their usage.
       (Bug #32197197)

     * Based on the terminology changes in WL#14189
       (https://dev.mysql.com/worklog/task/?id=14189), AdminAPI
       has been aligned with the new terms. Error and log
       messages now use the terms source (previously master) and
       replica (previously slave). (Bug #32152133)

     * During a Cluster.rebootClusterFromCompleteOutage()
       operation, the GTID superset is used to detect which
       instance should be used to reboot the cluster. If an
       instance had a diverging GTID set and you wanted to
       explicitly remove it from the cluster, the operation
       blocked because it could not determine which instance had
       the GTID superset. Previously, in such a situation there
       was no way to exclude the instance from the instances
       used to detect the GTID superset. Now, if you answer no
       during the interactive wizard, or configure the
       removeInstances option, the instance is not checked as
       part of finding the GTID superset. (Bug #32112864)

     * When an instance had left a ReplicaSet, and then its
       configuration was changed in a way that made it invalid
       for InnoDB ReplicaSet usage, the
       ReplicaSet.rejoinInstance() operation did not detect that
       the configuration was invalid. Now, instances are checked
       to ensure they are valid before rejoining them to a
       ReplicaSet. (Bug #31975416)

     * When upgrading the metadata using dba.upgradeMetadata(),
       if there are MySQL Router instances that need to be
       upgraded, the operation waits until all instances are
       upgraded before continuing. The operation offers you an
       option to re-check for outdated MySQL Router instances
       and continue with the metadata upgrade. A MySQL Router
       upgrade is only complete after a restart of the
       application, however the message printed did not mention
       that. This message now includes the information that
       MySQL Router instances must be restarted after the
       binaries are upgraded. (Bug #31882876)

     * When you were connected to a secondary instance,
       attempting to issue operations such as
       Cluster.rejoinInstance(), Cluster.addInstance(),
       Cluster.dissolve() and so on would fail. Now, AdminAPI
       always connects to the current primary.
       As part of this work the following changes were made:

          + Now, in the event that dba.createCluster() or
            Cluster.addInstance() fail with a Group Replication
            error, AdminAPI returns the
            performance_schema.error_log entries.

          + The Cluster.rejoinInstance() operation has been
            changed to succeed if the instance is already in the
            cluster, instead of throwing an exception.

          + The dba.rebootCluster() operation has been changed
            to not clear super_read_only on the instance.
       (Bug #31757737)

     * As part of the default settings for InnoDB Cluster, to
       ensure that instances automatically rejoin the cluster,
       the group_replication_start_on_boot option is
       automatically set to true. However, this meant that in
       environments with an external tool managing the cluster
       life cycle, for example an orchestrator such as
       Kubernetes, the automatic enabling of rejoin could cause
       conflicts with the tool. In addition, if the automatic
       rejoining of an instance was enabled at an unsuitable
       time (for example when rebooting, or while repairing a
       split-brain, and so on), a deadlock or long freezes could
       occur until a timeout happened. In some situations,
       instances could even potentially join the wrong cluster
       during a reconfiguration.
       To avoid such situations, the manualStartOnBoot boolean
       option has been added, which defaults to false. To
       disable the automatic rejoining of an instance, for
       example while repairing a split-brain, set the
       manualStartOnBoot option to true. This prevents the
       instance rejoining the cluster automatically while you
       make changes. You then need to rejoin the instance to the
       cluster manually, before setting the manualStartOnBoot
       option back to false to ensure instance it rejoins the
       cluster automatically again. Similarly, if you are using
       an external orchestrator to manage the life cycle of
       instances, set the manualStartOnBoot option to true
       across the whole cluster, to disable the automatic
       rejoining of instances to the cluster. Your orchestrator
       should then be configured to rejoin the instances
       manually. (Bug #31643595)

     * Calling dba.checkInstanceConfiguration() with verifyMyCnf
       set to a file which did not exist, the operation
       completed successfully saying the configuration file had
       been checked. The fix checks if the file specified by
       verifyMyCnf exists, prints an error if not, and ensures
       the console does not show unnecessary error messages.
       (Bug #31468546)

     * On an instance with the sql_mode variable set to
       ANSI_QUOTES, attempting to upgrade the metadata schema
       with dba.upgradeMetadata() failed with the error: Unknown
       column ‘MySQL Router’ in ‘field list’. This was related
       to a query which uses single quotes to quote strings. As
       part of this fix, the upgrade metadata operation now
       prepares the session to be used by AdminAPI, and amongst
       other sanity checks it ensures that the sql_mode for that
       session uses the default value to avoid incompatible user
       configured settings. Additionally, the same was done for
       the dba.getCluster() and dba.dropMetadataSchema()
       operations. (Bug #31428813)

     * If the MySQL Shell global session was connected to a
       sandbox instance, and that instance was stopped, MySQL
       Shell tried to incorrectly reconnect to the instance.
       Now, if the active session is connected to a sandbox
       instance which is being stopped, MySQL Shell closes the
       session. (Bug #31113914)

     * The output of Cluster.status() now includes additional
       information about instances that are registered in the
       metadata but not currently online. MySQL Shell now
       connects to offline instances found in the metadata and
       attempts to diagnose them, providing additional
       information such as their connectivity and status. (Bug
       #30501615)

     * Instances that are part of the underlying group but are
       not identified in the metadata, for example because they
       were configured manually and bypassing MySQL Shell, or
       because they were previously removed from the InnoDB
       Cluster but were not properly decommissioned, are now
       shown in the output of Cluster.status(), along with
       diagnostic warnings about the metadata discrepancy. This
       ensures you can detect situations where an instance is
       participating in the group but is not being managed by
       MySQL Shell. (Bug #27882663)

     * An instance that belongs to an InnoDB Cluster is
       identified by its server UUID. If the UUID changed after
       the instance had left the cluster, for example because
       you used MySQL Enterprise Backup to restore from a
       backup, then the instance could not be rejoined to the
       cluster. Now, if the cluster encounters this situation,
       it checks the metadata to see if the instance can be
       identified using its host and port. If found, the
       metadata is updated based on the options used for the
       rejoin operation. This check is executed during the
       Cluster.rejoinInstance() and Cluster.rescan() operations.
       Additionally, a check is executed to verify the serverId
       of all the instances is registered in the metadata as an
       instance attribute. If it is not, the metadata is updated
       accordingly. This check is executed on add, rejoin and
       rescan operations. (Bug #26649039)

Functionality Added or Changed


     * MySQL Shell’s parallel table import utility can now
       import a specified list of input data files, and it
       supports wildcard pattern matching to include all
       relevant files from a location. Multiple files uploaded
       by a single run of the utility are placed into a single
       relational table, so for example, data that has been
       exported from multiple hosts and stored in multiple files
       could be merged into a single table to be used for
       analytics. The files can be compressed in the gzip or
       zstd format, and in that case the utility reads them from
       storage in the compressed format, saving bandwidth for
       that part of the transfer. The utility then uses its
       parallel connections to decompress and upload several
       files simultaneously to the target server.

Bugs Fixed


     * When MySQL Shell’s instance dump utility
       util.dumpInstance() was run with the ocimds option set to
       true to check compatibility with MySQL Database Service,
       and the users option set to true to include users and
       their roles and grants in the dump, the utility reported
       some compatibility errors for privileges that actually
       were permitted. MySQL Shell’s allowed list of privileges
       for MySQL Database Service has now been updated. (Bug
       #32213605)

     * The behavior of MySQL Shell’s table dump utility
       util.dumpTables() and dump loading utility
       util.loadDump() regarding the schemas for single table
       dumps and loads has been changed. Previously, the dump
       files produced for a single table did not contain the SQL
       statements to recreate the schema, so the schema had to
       exist in the target MySQL instance before the dump
       loading utility could load the table. Now, the dumps
       produced by the table dump utility contain the schema
       statements, and when they are loaded with the dump
       loading utility, by default, the schema is created in the
       target MySQL instance if it does not already exist. The
       schema option can be used to load the table dump into
       another schema that exists in the target MySQL instance.
       Table dumps created using the earlier version of the
       utility still require the schema option and an existing
       schema. (Bug #32165101)

     * MySQL Shell’s table dump utility util.dumpTables() now
       supports the ocimds, compatibility, ociParManifest, and
       ociParExpireTime options, so you can check compatibility
       with MySQL Database Service, and generate
       pre-authenticated request URLs for the dump files. Also,
       the ignoreVersion option has been extended to allow the
       import of a dump that was created without the ocimds
       option into a MySQL DB System. (Bug #32140970)

     * If a dump included users that were created with external
       authentication plugins, MySQL Shell’s dump loading
       utility util.loadDump() was unable to load the dump if
       those plugins were not available on the target server
       instance. The ocimds option for MySQL Shell’s instance
       dump utility util.dumpInstance() and schema dump utility
       util.dumpSchemas which checks compatibility with MySQL
       Database Service, now checks for accounts using
       authentication plugins that are not supported in MySQL
       Database Service. The compatibility option has an
       additional modification option skip_invalid_accounts,
       which removes such user accounts. (Bug #32115948)

     * Previously, MySQL Shell’s dump loading utility
       util.loadDump() stopped with an error if the loadUsers
       option was set to true but the supplied dump files did
       not contain user accounts. The utility now displays a
       warning and continues in this situation. (Bug #32115861)

     * MySQL Shell’s instance dump utility util.dumpInstance(),
       schema dump utility util.dumpSchemas(), and table dump
       utility util.dumpTables() falls back to using the LOCK
       TABLES privilege to lock dumped tables if the consistent
       option is set to true, which is the default, and the
       RELOAD privilege is not available. However, the locking
       operation could cause an implicit commit on active
       transactions, meaning that the data was not dumped
       consistently. The locking has now been corrected to
       ensure consistency in this situation. (Bug #32107327, Bug
       #101410)

     * When MySQL Shell’s dump loading utility util.loadDump()
       used indexes to identify row boundaries, an error
       occurred if an index pointed beyond the data in the read
       buffer. The utility now checks for this situation and
       ignores the index if so. (Bug #32072961)

     * When MySQL Shell was attempting to reconnect to a server,
       Ctrl + C did not interrupt the operation. The interrupt
       now functions and sets the retry attempts counter to zero
       so that the sequence exits correctly. (Bug #32041342)

     * MySQL Shell can now be built using Python 3.9. (Bug
       #32020230)

     * The updateGtidSet option for MySQL Shell’s dump loading
       utility util.loadDump() could not be used with MySQL DB
       System due to a permissions restriction. The utility now
       uses a stored procedure that is permitted, so the option
       can be used. (Bug #32009225)

     * When MySQL Shell’s instance dump utility
       util.dumpInstance(), schema dump utility
       util.dumpSchemas(), or table dump utility
       util.dumpTables() was exporting to an Oracle Cloud
       Infrastructure Object Storage bucket, if there was a loss
       of connectivity or routing to the Object Storage server,
       MySQL Shell stopped unexpectedly. The error is now
       handled correctly. (Bug #32005418)

     * MySQL Shell’s dump loading utility util.loadDump()
       returned an exception if a header value in a response was
       empty. (Bug #31979374)

     * MySQL Shell did not initialize Python 3.8’s new
       cf_feature_version compiler flag field, which could cause
       an exception when format strings were used. (Bug
       #31926697)

     * Where MySQL Shell is using a system installation of
       Python rather than the bundled version, the minimum
       version that MySQL Shell supports is now Python 3.6.
       Python 3.4.3 was the previous minimum for a system
       installation. The bundled version is Python 3.7.7. (Bug
       #31900744)

     * MySQL Shell’s instance dump utility util.dumpInstance(),
       schema dump utility util.dumpSchemas(), and table dump
       utility util.dumpTables() use table statistics to
       identify a suitable default row size. If the statistics
       for a table are outdated or not present, this can cause
       issues for the chunking process. In this situation, MySQL
       Shell now issues a message to suggest using an ANALYZE
       TABLE statement to produce up to date statistics. (Bug
       #31766490)

     * The skipBinlog option for MySQL Shell’s dump loading
       utility util.loadDump() skips binary logging on the
       target MySQL instance for the import. The option is not
       suitable for MySQL DB System as the binary logging status
       cannot be changed, and the import now fails with an error
       message if the option is used in that situation. For
       other MySQL instances, the utility now checks whether the
       user has the required privileges to set the sql_log_bin
       system variable, and fails with an error message if they
       do not. (Bug #31748786)

     * MySQL Shell’s instance dump utility util.dumpInstance(),
       schema dump utility util.dumpSchemas(), and table dump
       utility util.dumpTables() ordered the data fetched for
       export using the first column of a unique index for the
       table. The same method was used to query data for
       chunking purposes. The utilities now use all columns of
       the unique index for ordering. In addition, performance
       is improved by the addition of a cache to store
       frequently-used instance metadata. The cache is populated
       for all the schema objects at once, rather than by
       individual queries as needed. (Bug #31706755)

     * MySQL Shell’s disconnect function was added to the shell
       global object. (Bug #31704380)

On Behalf of Oracle/MySQL Release Engineering
Prashant Tekriwal

X DevAPI Traffic Compression With Connector/J

X Protocol traffic compression is available on MySQL Server since version 8.0.19. A connector that also supports compression on its end can leverage this feature and reduce the byte streams that are exchanged with the Server.

By default, connections to a MySQL server are uncompressed, thus permitting exchanging data with a client or connector that doesn’t support compression. However, given a client or connector that also supports compression, it is recommended that client and server negotiate the connection compression by default. If this negotiation concludes successfully, both ends can then compress the data they send.

Compression at this level allows reducing the amount of bytes exchanged over the network, but at the cost of additional CPU resources required to run data inflate and deflate operations. The benefits of compression, therefore, occur primarily on low network bandwidth. One can assess the gain or loss due to the compression only after measuring properly the average traffic sizes for a period of time for both the compressed and uncompressed connections.

Connector/J version 8.0.20 came out with basic support for traffic compression over X Protocol connections. For this purpose, a new connection option: xdevapi.compression, was introduced. As of Connector/J 8.0.22, this feature was leveled up with the introduction of two additional connection options: xdevapi.compression-extensions and xdevapi.compression-algorithms.

Let it be clear that the new compression features in the X DevAPI has no impact whatsoever on the existing compression behavior or settings of Connector/J JDBC implementation.

Continue reading