X DevAPI Traffic Compression With Connector/J

Posted on by

X Protocol traffic compression is available on MySQL Server since version 8.0.19. A connector that also supports compression on its end can leverage this feature and reduce the byte streams that are exchanged with the Server.

By default, connections to a MySQL server are uncompressed, thus permitting exchanging data with a client or connector that doesn’t support compression. However, given a client or connector that also supports compression, it is recommended that client and server negotiate the connection compression by default. If this negotiation concludes successfully, both ends can then compress the data they send.

Compression at this level allows reducing the amount of bytes exchanged over the network, but at the cost of additional CPU resources required to run data inflate and deflate operations. The benefits of compression, therefore, occur primarily on low network bandwidth. One can assess the gain or loss due to the compression only after measuring properly the average traffic sizes for a period of time for both the compressed and uncompressed connections.

Connector/J version 8.0.20 came out with basic support for traffic compression over X Protocol connections. For this purpose, a new connection option: xdevapi.compression, was introduced. As of Connector/J 8.0.22, this feature was leveled up with the introduction of two additional connection options: xdevapi.compression-extensions and xdevapi.compression-algorithms.

Let it be clear that the new compression features in the X DevAPI has no impact whatsoever on the existing compression behavior or settings of Connector/J JDBC implementation.

Continue reading →

MySQL Connector/ODBC 8.0.22 has been released

Posted on by

Dear MySQL users,

MySQL Connector/ODBC 8.0.22 is a new version in the MySQL Connector/ODBC 8.0 series, the ODBC driver for the MySQL Server.

The available downloads include both a Unicode driver and an ANSI driver based on the same modern codebase. Please select the driver type you need based on the type of your application – Unicode or ANSI. Server-side prepared statements are enabled by default. It is suitable for use with the latest MySQL server version 8.0.

This release of the MySQL ODBC driver is conforming to the ODBC 3.8 specification. It contains implementations of key 3.8 features, including self-identification as a ODBC 3.8 driver, streaming of out for binary types only), and support of the SQL_ATTR_RESET_CONNECTION connection attribute (for the Unicode driver only).

The release is now available in source and binary form for a number of platforms from our download pages at

https://dev.mysql.com/downloads/connector/odbc/

For information on installing, please see the documentation at

https://dev.mysql.com/doc/connector-odbc/en/connector-odbc-installation.html

Enjoy and thanks for the support!

================================================================

Changes in MySQL Connector/ODBC 8.0.22 (2020-10-19, General Availability)

Functionality Added or Changed

  • For enhanced security of the existing ENABLE_LOCAL_INFILE connection string option, the new ENABLE_LOCAL_DIR option allows restricting LOCAL data loading to files located i this designated directory. Example usage:
 // LOAD LOCAL DATA DIR FROM /tmp
SQLRETURN rc =
  SQLDriverConnect(
    hdbc1,NULL,
     "DSN=myDSN;UID=root;PWD=pwd;DATABASE=test;LOAD_DATA_LOCAL_DIR=/tmp",
    SQL_NTS, conn_out, sizeof(conn_out), &conn_out_len,
    SQL_DRIVER_NOPROMPT);

// LOAD LOCAL DATA FROM EVERYWHERE
SQLRETURN rc =
  SQLDriverConnect(
    hdbc1,NULL,
    "DSN=myDSN;UID=root;PWD=pwd;DATABASE=test;ENABLE_LOCAL_INFILE=1;",
    SQL_NTS, conn_out, sizeof(conn_out), &conn_out_len,
    SQL_DRIVER_NOPROMPT);
  • Connections made using the MySQL Enterprise Edition SAS LDAP authentication plugin now are supported on Windows and Linux, but not on macOS. Connector/ODBC implement the SCRAM-SHA-1 authentication method of the SASL authentication protocol.

Bugs Fixed

  • Fixed an issue where a parameterized query could cause memory corruption. (Bug #31678876, Bug #100329)
  • Under some circumstances when using server-side prepared statements, the first row of a multi-row match was not returned with the result; while it was returned when using client-side prepared statements instead. (Bug #31373948, Bug #95423)
  • Inserting binary data (BLOBs) using SQLPutData() would report a syntax error. (Bug #31349038)

On Behalf of MySQL Release Engineering Team,
Kent Boortz

MySQL Connector/Node.js 8.0.22 has been released

Posted on by

Dear MySQL users,

MySQL Connector/Node.js is a new Node.js driver for use with the X
DevAPI. This release, v8.0.22, is a maintenance release of the
MySQL Connector/Node.js 8.0 series.

The X DevAPI enables application developers to write code that combines
the strengths of the relational and document models using a modern,
NoSQL-like syntax that does not assume previous experience writing
traditional SQL.

MySQL Connector/Node.js can be downloaded through npm (see
  https://www.npmjs.com/package/@mysql/xdevapi for details) or from
  https://dev.mysql.com/downloads/connector/nodejs/.

To learn more about how to write applications using the X DevAPI, see
  http://dev.mysql.com/doc/x-devapi-userguide/en/.
For more information about how the X DevAPI is implemented in MySQL
Connector/Node.js, and its usage, see
  http://dev.mysql.com/doc/dev/connector-nodejs/.

Please note that the X DevAPI requires at least MySQL Server version
8.0 or higher with the X Plugin enabled. For general documentation
about how to get started using MySQL as a document store, see
  http://dev.mysql.com/doc/refman/8.0/en/document-store.html.


Changes in MySQL Connector/Node.js 8.0.22 (2020-10-19, General Availability)

Functionality Added or Changed

     * Improved test execution configuration to better align
       with other connectors. For example, unified environment
       variable names (such as changing NODE_TEST_MYSQL_HOST to
       MYSQLX_HOST). See the Connector/Node.js documentation
       (https://dev.mysql.com/doc/dev/connector-nodejs/8.0/) for
       usage information.


Bugs Fixed

     * Non-BIGINT values stored in BIGINT columns were not
       decoded properly in result sets. (Bug #31686805, Bug
       #100324)

     * Fetched results from a SET column would only contain one
       value from the set. (Bug #31654667, Bug #100255)

     * Deprecated the dbPassword and dbUser property names;
       which were aliases to the password and user properties.
       Their usage now emits deprecation level errors. (Bug
       #31599660)

     * Added a SERVER_GONE error handler to avoid potential
       circular dependency warnings with Node.js >= 14.0.0. (Bug
       #31586107, Bug #99869)

     * Restricted the offset() method to the CollectionFind and
       TableSelect APIs, as described in the X DevAPI
       specification. Using offset() on other APIs yielded this
       error: “Error: The server has gone away”. Instead, this
       intended behavior is available by using a combination of
       “sort()” or “orderBy()” and “limit()”. (Bug #31418813)

     * The nextResult() method returned false against an empty
       result set, and now returns true. Alternatively, use
       hasData() to check if a result set has data. (Bug
       #31037211)

     * The column.getType() method now returns the stringified
       type identifier when before it returned its numeric
       value. For example, DATETIME is now returned instead of
       12. (Bug #30922711)

     * Improved memory management for work performed by 3rd
       party APIs. (Bug #30845472)

     * Added support for lazy decoding of binary column metadata
       content. (Bug #30845366)

On Behalf of Oracle/MySQL Release Engineering Team,

Hery Ramilison

MySQL Connector/C++ 8.0.22 has been released

Posted on by

Dear MySQL users,

MySQL Connector/Node.js is a new Node.js driver for use with the X
DevAPI. This release, v8.0.22, is a maintenance release of the
MySQL Connector/Node.js 8.0 series.

The X DevAPI enables application developers to write code that combines
the strengths of the relational and document models using a modern,
NoSQL-like syntax that does not assume previous experience writing
traditional SQL.

MySQL Connector/Node.js can be downloaded through npm (see
  https://www.npmjs.com/package/@mysql/xdevapi for details) or from
  https://dev.mysql.com/downloads/connector/nodejs/.

To learn more about how to write applications using the X DevAPI, see
  http://dev.mysql.com/doc/x-devapi-userguide/en/.
For more information about how the X DevAPI is implemented in MySQL
Connector/Node.js, and its usage, see
  http://dev.mysql.com/doc/dev/connector-nodejs/.

Please note that the X DevAPI requires at least MySQL Server version
8.0 or higher with the X Plugin enabled. For general documentation
about how to get started using MySQL as a document store, see
  http://dev.mysql.com/doc/refman/8.0/en/document-store.html.


Changes in MySQL Connector/Node.js 8.0.22 (2020-10-19, General Availability)

Functionality Added or Changed

     * Improved test execution configuration to better align
       with other connectors. For example, unified environment
       variable names (such as changing NODE_TEST_MYSQL_HOST to
       MYSQLX_HOST). See the Connector/Node.js documentation
       (https://dev.mysql.com/doc/dev/connector-nodejs/8.0/) for
       usage information.


Bugs Fixed

     * Non-BIGINT values stored in BIGINT columns were not
       decoded properly in result sets. (Bug #31686805, Bug
       #100324)

     * Fetched results from a SET column would only contain one
       value from the set. (Bug #31654667, Bug #100255)

     * Deprecated the dbPassword and dbUser property names;
       which were aliases to the password and user properties.
       Their usage now emits deprecation level errors. (Bug
       #31599660)

     * Added a SERVER_GONE error handler to avoid potential
       circular dependency warnings with Node.js >= 14.0.0. (Bug
       #31586107, Bug #99869)

     * Restricted the offset() method to the CollectionFind and
       TableSelect APIs, as described in the X DevAPI
       specification. Using offset() on other APIs yielded this
       error: “Error: The server has gone away”. Instead, this
       intended behavior is available by using a combination of
       “sort()” or “orderBy()” and “limit()”. (Bug #31418813)

     * The nextResult() method returned false against an empty
       result set, and now returns true. Alternatively, use
       hasData() to check if a result set has data. (Bug
       #31037211)

     * The column.getType() method now returns the stringified
       type identifier when before it returned its numeric
       value. For example, DATETIME is now returned instead of
       12. (Bug #30922711)

     * Improved memory management for work performed by 3rd
       party APIs. (Bug #30845472)

     * Added support for lazy decoding of binary column metadata
       content. (Bug #30845366) Dear MySQL users,

MySQL Connector/C++ 8.0.22 is a new release version of the MySQL
Connector/C++ 8.0 series.

Connector/C++ 8.0 can be used to access MySQL implementing Document
Store or in a traditional way, using SQL queries. It allows writing
both C++ and plain C applications using X DevAPI and X DevAPI for C.
It also supports the legacy API of Connector/C++ 1.1 based on JDBC4.

To learn more about how to write applications using X DevAPI, see
“X DevAPI User Guide” at

  https://dev.mysql.com/doc/x-devapi-userguide/en/

See also “X DevAPI Reference” at

  https://dev.mysql.com/doc/dev/connector-cpp/devapi_ref.html

and “X DevAPI for C Reference” at

  https://dev.mysql.com/doc/dev/connector-cpp/xapi_ref.html

For generic information on using Connector/C++ 8.0, see

  https://dev.mysql.com/doc/dev/connector-cpp/

For general documentation about how to get started using MySQL
as a document store, see

  http://dev.mysql.com/doc/refman/8.0/en/document-store.html

To download MySQL Connector/C++ 8.0.22, see the “General Availability (GA)
Releases” tab at

  https://dev.mysql.com/downloads/connector/cpp/


Changes in MySQL Connector/C++ 8.0.22 (2020-10-19, General Availability)

Compilation Notes

     * Connector/C++ now can be compiled using MinGW on Windows.
       Thanks to Eric Beuque for the contribution. Note that
       this enables building on MinGW but does not make MinGW an
       officially supported platform for Connector/C++. (Bug
       #31636723, Bug #100248)


Connection Management Notes

     * For connections made using X Plugin, Connector/C++ now
       enables specifying the compression algorithms to be used
       for connections that use compression. Connection URIs and
       SessionSettings objects permit explicitly specifying the
       preferred algorithms:

          + URI strings permit a compression-algorithms option.
            The value is an algorithm name, or a list of one or
            more comma-separated algorithms specified as an
            array. Examples:
            mysqlx://user:password@host:port/db?compression-algorithms=lz4
            mysqlx://user:password@host:port/db?compression-algorithms=[lz4,zstd_stream]

          + SessionSettings objects permit a
            SessionOption::COMPRESSION_ALGORITHMS option. The
            value is a list of one or more comma-separated
            algorithms. Examples:
            mysqlx::Session sess(SessionOption::USER, “user_name”,
                     SessionOption::PWD, “password”,
                     SessionOption::COMPRESSION_ALGORITHMS, “lz4”);
            mysqlx::Session sess(SessionOption::USER, “user_name”,
                     SessionOption::PWD, “password”,
                     SessionOption::COMPRESSION_ALGORITHMS, “lz4,zstd_stream”);

            Alternatively, the algorithms value can be given as
            a container:
            std::list<std::string> algorithms = {“lz4″,”zstd_stream”};
            mysqlx::Session sess(SessionOption::USER, “user_name”,
                     SessionOption::PWD, “password”,
                     SessionOption::COMPRESSION_ALGORITHMS, algorithms);

          + For X DevAPI for C, there is a new
            MYSQLX_OPT_COMPRESSION_ALGORITHMS option and
            corresponding OPT_COMPRESSION_ALGORITHMS helper
            macro.
            URI mode follows X DevAPI URI mode:
            mysqlx_session_t *sess = mysqlx_get_session_from_url(
            “mysqlx://user:password@host:port/db?compression-algorithms=[lz4,zstd_stream]”, &error);

            Option mode follows the string format used for
            SessionOption:
            mysqlx_session_option_set(opt,
                      OPT_HOST(“host_name”),
                      OPT_USER(“user”),
                      OPT_PWD(“password”),
                      OPT_COMPRESSION_ALGORITHMS(“lz4,zstd_stream”),
                      PARAM_END));

       These rules apply:

          + Permitted algorithm names are zstd_stream,
            lz4_message, and deflate_stream, and their aliases
            zstd, lz4, and deflate. Names are case-insensitive.
            Unknown names are ignored.

          + Compression algorithms options permit multiple
            algorithms, which should be listed in priority
            order. Options that specify multiple algorithms can
            mix full algorithm names and aliases.

          + If no compression algorithms option is specified,
            the default is
            zstd_stream,lz4_message,deflate_stream.

          + The actual algorithm used is the first of those
            listed in the compression algorithms option that is
            also permitted on the server side. However, the
            option for compression algorithms is subject to the
            compression mode:
               o If the compression mode is disabled, the
                 compression algorithms option is ignored.
               o If the compression mode is preferred but no
                 listed algorithm is permitted on the server
                 side, the connection is uncompressed.
               o If the compression mode is required but no
                 listed algorithm is permitted on the server
                 side, an error occurs.
       See also Connection Compression with X Plugin
       (https://dev.mysql.com/doc/refman/8.0/en/x-plugin-connection-compression.html).


Legacy (JDBC API) Notes

     * For applications that use the legacy JDBC API (that is,
       not X DevAPI or X DevAPI for C), Connector/C++ binary
       distributions now include the libraries that provide the
       client-side LDAP authentication plugins, as well as any
       dependent libraries required by the plugins. This enables
       Connector/C++ application programs to connect to MySQL
       servers using simple LDAP authentication, or SASL LDAP
       authentication using the SCRAM-SHA-1 authentication
       method.
       Note
       LDAP authentication requires use of a server from a MySQL
       Enterprise Edition distribution. For more information
       about the LDAP authentication plugins, see LDAP Pluggable
       Authentication
       (https://dev.mysql.com/doc/refman/8.0/en/ldap-pluggable-authentication.html).
       If Connector/C++ was installed from a compressed tar file
       or Zip archive, the application program will need to set
       the OPT_PLUGIN_DIR connection option to the appropriate
       directory so that the bundled plugin library can be
       found. (Alternatively, copy the required plugin library
       to the default directory expected by the client library.)
       Example:
            sql::ConnectOptionsMap connection_properties;

            // To use simple LDAP authentication …

            connection_properties[“userName”] = “simple_ldap_user_name”;
            connection_properties[“password”] = “simple_ldap_password”;
            connection_properties[OPT_ENABLE_CLEARTEXT_PLUGIN]=true;

            // To use SASL LDAP authentication using SCRAM-SHA-1 …

            connection_properties[“userName”] = “sasl_ldap_user_name”;
            connection_properties[“password”] = “sasl_ldap_scram_password”;

            // Needed if Connector/C++ was installed from tar file or Zip archive …

            connection_properties[OPT_PLUGIN_DIR] = “${INSTALL_DIR}/lib{64}/plugin”;

            auto *driver = get_driver_instance();
            auto *con = driver->connect(connection_properties);

            // Execute statements …

            con->close();

     * For applications that use the legacy JDBC API (that is,
       not X DevAPI or X DevAPI for C), LOCAL data loading
       capability for the LOAD DATA statement previously could
       be controlled on the client side only by enabling it for
       all files accessible to the client, or by disabling it
       altogether. The new OPT_LOAD_DATA_LOCAL_DIR option
       enables restricting LOCAL data loading to files located
       in a designated directory. For example, to set the value
       at connect time:
            sql::ConnectOptionsMap opt;
            opt[OPT_HOSTNAME] = “localhost”;
            opt[OPT_LOAD_DATA_LOCAL_DIR] = “/tmp”;

            sql::Connection *conn = driver->connect(opt);

       OPT_LOAD_DATA_LOCAL_DIR can also be set after connect
       time:
            sql::ConnectOptionsMap opt;
            opt[OPT_HOSTNAME] = “localhost”;

            sql::Connection *conn = driver->connect(opt);

            //…. some queries / inserts / updates

            std::string path= “/tmp”;
            conn->setClientOption(OPT_LOAD_DATA_LOCAL_DIR, path);

            // LOAD LOCAL DATA DIR …

            //Disable LOCAL INFILE by setting to null
            conn->setClientOption(OPT_LOAD_DATA_LOCAL_DIR, nullptr);

       The OPT_LOAD_DATA_LOCAL_DIR option maps onto the
       MYSQL_OPT_LOAD_DATA_LOCAL_DIR option for the
       mysql_options() C API function. For more information, see
       Security Considerations for LOAD DATA LOCAL
       (https://dev.mysql.com/doc/refman/8.0/en/load-data-local-security.html).


Bugs Fixed

     * String decoding failed for utf-8 strings that began with
       a \xEF byte-order mark. (Bug #31656092)

     * With the CLIENT_MULTI_FLAG option enabled, executing
       multiple statements in a batch caused the next query to
       fail with a Commands out of sync error. (Bug #31399362)

     * For connections made using X Plugin, connections over
       Unix socket files did not work. (Bug #31329938)

     * For connections made using X Plugin, the default
       compression mode was DISABLED rather than PREFERRED. (Bug
       #31173447) Dear MySQL users,

MySQL Connector/Node.js is a new Node.js driver for use with the X
DevAPI. This release, v8.0.22, is a maintenance release of the
MySQL Connector/Node.js 8.0 series.

The X DevAPI enables application developers to write code that combines
the strengths of the relational and document models using a modern,
NoSQL-like syntax that does not assume previous experience writing
traditional SQL.

MySQL Connector/Node.js can be downloaded through npm (see
  https://www.npmjs.com/package/@mysql/xdevapi for details) or from
  https://dev.mysql.com/downloads/connector/nodejs/.

To learn more about how to write applications using the X DevAPI, see
  http://dev.mysql.com/doc/x-devapi-userguide/en/.
For more information about how the X DevAPI is implemented in MySQL
Connector/Node.js, and its usage, see
  http://dev.mysql.com/doc/dev/connector-nodejs/.

Please note that the X DevAPI requires at least MySQL Server version
8.0 or higher with the X Plugin enabled. For general documentation
about how to get started using MySQL as a document store, see
  http://dev.mysql.com/doc/refman/8.0/en/document-store.html.


Changes in MySQL Connector/Node.js 8.0.22 (2020-10-19, General Availability)

Functionality Added or Changed

     * Improved test execution configuration to better align
       with other connectors. For example, unified environment
       variable names (such as changing NODE_TEST_MYSQL_HOST to
       MYSQLX_HOST). See the Connector/Node.js documentation
       (https://dev.mysql.com/doc/dev/connector-nodejs/8.0/) for
       usage information.


Bugs Fixed

     * Non-BIGINT values stored in BIGINT columns were not
       decoded properly in result sets. (Bug #31686805, Bug
       #100324)

     * Fetched results from a SET column would only contain one
       value from the set. (Bug #31654667, Bug #100255)

     * Deprecated the dbPassword and dbUser property names;
       which were aliases to the password and user properties.
       Their usage now emits deprecation level errors. (Bug
       #31599660)

     * Added a SERVER_GONE error handler to avoid potential
       circular dependency warnings with Node.js >= 14.0.0. (Bug
       #31586107, Bug #99869)

     * Restricted the offset() method to the CollectionFind and
       TableSelect APIs, as described in the X DevAPI
       specification. Using offset() on other APIs yielded this
       error: “Error: The server has gone away”. Instead, this
       intended behavior is available by using a combination of
       “sort()” or “orderBy()” and “limit()”. (Bug #31418813)

     * The nextResult() method returned false against an empty
       result set, and now returns true. Alternatively, use
       hasData() to check if a result set has data. (Bug
       #31037211)

     * The column.getType() method now returns the stringified
       type identifier when before it returned its numeric
       value. For example, DATETIME is now returned instead of
       12. (Bug #30922711)

     * Improved memory management for work performed by 3rd
       party APIs. (Bug #30845472)

     * Added support for lazy decoding of binary column metadata
       content. (Bug #30845366)

On Behalf of Oracle/MySQL Release Engineering Team,

Hery Ramilison

MySQL Connector/Python 8.0.22 has been released

Posted on by

Dear MySQL users,

MySQL Connector/Python 8.0.22 is the latest GA release version of the
MySQL Connector Python 8.0 series. The X DevAPI enables application
developers to write code that combines the strengths of the relational
and document models using a modern, NoSQL-like syntax that does not
assume previous experience writing traditional SQL.

To learn more about how to write applications using the X DevAPI, see

http://dev.mysql.com/doc/x-devapi-userguide/en/

For more information about how the X DevAPI is implemented in MySQL
Connector/Python, and its usage, see

http://dev.mysql.com/doc/dev/connector-python

Please note that the X DevAPI requires at least MySQL Server version 8.0
or higher with the X Plugin enabled. For general documentation about how
to get started using MySQL as a document store, see

http://dev.mysql.com/doc/refman/8.0/en/document-store.html

To download MySQL Connector/Python 8.0.22, see the “General Availability
(GA) Releases” tab at

http://dev.mysql.com/downloads/connector/python/

Enjoy!

Changes in MySQL Connector/Python 8.0.22 (2020-10-19, General Availability)

Functionality Added or Changed


     * Added Django 3.0 support while preserving compatibility with
       Django 2.2. Removed support for Django 1.11 with Python 2.7.

     * Previously, the client-side mysql_clear_password authentication
       plugin was not supported. Now, it is permitted to send passwords
       without hashing or encryption by using mysql_clear_password on
       the client side together with any server-side plugin that needs a
       clear text password, such as for LDAP pluggable authentication.
       Connector/Python returns an exception if the mysql_clear_password
       plugin is requested but the connection is neither encrypted nor
       using Unix domain sockets. For usage information, see Client-Side
       Cleartext Pluggable Authentication
(https://dev.mysql.com/doc/refman/8.0/en/cleartext-pluggable-authentication.html).

     * Connections made using the MySQL Enterprise Edition SASL LDAP
       authentication plugin now are supported on Windows and Linux, but
       not on macOS. Connector/Python implements the SCRAM-SHA-1
       authentication method of the SASL authentication protocol.

     * The new compression-algorithms connection option sets the order
       by which supported algorithms are negotiated and selected to send
       compressed data over X Protocol connections. The algorithms
       available are specified by the server and currently include:
       lz4_message, deflate_stream, and zstd_stream. Supported algorithm
       aliases are lz4, deflate, and zstd. Unknown or unsupported values
       are ignored.  Example usage:
session = mysqlx.get_session({
    “host”: “localhost”,
    “port”: 33060,
    “user”: “root”,
    “password”: “s3cr3t”,
    “compression”: “required”,
    “compression-algorithms”: [“lz4′, “zstd_stream”]
})


     * For enhanced security of the existing allow_local_infile
       connection string option, the new allow_local_infile_in_path
       option allows restricting LOCAL data loading to files located in
       this designated directory.

     * Refactored the Connector/Python build system by removing
       artifacts of old implementations, improved debugging, and now
       statically link the C extensions. This also exposes the distutils
       commands, to allow the end-user build packages.

     * The pure Python and C extension implementations were combined
       into a single package; this applies to both DEB and RPM packages.

Bugs Fixed


     * Fixed a memory leak in the C-extension implementation when using
       the Decimal data type. Thanks to Kan Liyong for the patch.
       (Bug #31335275, Bug #99517)

     * Copyright and License headers were missing in the Python modules
       generated by protoc. (Bug #31267800)

     * When creating an index on a collection, if a collation was
       specified but the field is not of the type TEXT, then an error
       message was generated with a wrong field type.  It’d always
       report it as GEOJSON. (Bug #27535063)

     * The reset connection command was missing from the C-extension
       implementation, which is required to reuse a connection from the
       pool. As such, connection pooling is now allowed with the
       C-extension implementation. (Bug #20811567, Bug #27489937)

Enjoy and thanks for the support!

On behalf of the MySQL Release Team,
Nawaz Nazeer Ahamed

MySQL Connector/NET 8.0.22 has been released

Posted on by

Dear MySQL users,

MySQL Connector/NET 8.0.22 is the latest General Availability release of
the MySQL Connector/NET 8.0 series. This version supports .NET 5.0
and the X DevAPI, which enables application developers to write code
that combines the strengths of the relational and document models using
a modern, NoSQL-like syntax that does not assume previous experience
writing traditional SQL.

To learn more about how to write applications using the X DevAPI, see
http://dev.mysql.com/doc/x-devapi-userguide/en/index.html. For more
information about how the X DevAPI is implemented in Connector/NET, see
http://dev.mysql.com/doc/dev/connector-net.
NuGet packages provide functionality at a project level. To get the
full set of features available in Connector/NET such as availability
in the GAC, integration with Visual Studio’s Entity Framework Designer
and integration with MySQL for Visual Studio, installation through
the MySQL Installer or the stand-alone MSI is required.

Please note that the X DevAPI requires at least MySQL Server version
8.0 or higher with the X Plugin enabled. For general documentation
about how to get started using MySQL as a document store, see
http://dev.mysql.com/doc/refman/8.0/en/document-store.html.

To download MySQL Connector/NET 8.0.22, see
http://dev.mysql.com/downloads/connector/net/

Installation instructions can be found at
https://dev.mysql.com/doc/connector-net/en/connector-net-installation.html

Changes in MySQL Connector/NET 8.0.22 (2020-10-19, General Availability)

* Functionality Added or Changed

* Bugs Fixed

Functionality Added or Changed

* Revisions to the MySql.Data.EntityFrameworkCore namespace
reduced the number of public classes. (Bug #31353208)

* The following synonyms for the Server connection string
option are deprecated in this release: address, addr, and
network address. (Bug #31248601)

* Previously, the client-side mysql_clear_password
authentication plugin was not supported. Now, it is
permitted to send passwords without hashing or encryption
by using mysql_clear_password on the client side together
with any server-side plugin that needs a clear text
password, such as for LDAP pluggable authentication.
Connector/NET returns an error if the
mysql_clear_password plugin is requested, but the
connection is neither encrypted nor using Unix domain
sockets. For usage information, see Client-Side Cleartext
Pluggable Authentication
(https://dev.mysql.com/doc/refman/8.0/en/cleartext-pluggable-authentication.html). (Bug #30340510)

* For enhanced security of the existing
AllowLoadLocalInfile connection string option, a single
folder that is safe to upload files from now can be
specified with the new AllowLoadLocalInfileInPath option
(see Options for Classic MySQL Protocol Only (https://dev.mysql.com/doc/connector-net/en/connector-net-8-0-connection-options.html#connector-net-8-0-connection-options-classic)).

* Connector/NET now supports Entity Framework 6.4, which
extends the compatibility of the provider to include the
Linux and macOS platforms when used with the Universal
Windows Platform (UWP) .NET implementation (see Entity
Framework 6 Support (https://dev.mysql.com/doc/connector-net/en/connector-net-entityframework60.html)).
Connector/NET continues to support the .NET Framework
implementation of Entity Framework.

* Connections made using the MySQL Enterprise Edition SASL
LDAP authentication plugin now are supported on Windows
and Linux, but not on macOS. Connector/NET implements the
SCRAM-SHA-1 authentication method of the SASL
authentication protocol.

* The new compression-algorithms connection option sets the
order by which supported algorithms are negotiated and
selected to send compressed data over X Protocol
connections (see Options for X Protocol Only (https://dev.mysql.com/doc/connector-net/en/connector-net-8-0-connection-options.html#connector-net-8-0-connection-options-xprotocol)).

* In addition to providing continued support for .NET Core
and .NET Framework, Connector/NET now includes support
for the new .NET 5.0 framework. Compatibility testing was
performed with the preview versions of .NET 5.0 and
Visual Studio to encourage the efforts of early adopters.
Among other capabilities, .NET 5.0 offers uniform runtime
behaviors and developer experiences by taking the best of
.NET Core, .NET Framework, Xamarin, and Mono.

Bugs Fixed

* Procedure names were malformed before being sent to the
server when the database name was not specified in the
connection string. (Bug #31669587, Bug #100306)

* Microseconds were deserialized incorrectly when
MySqlCommand.Prepare() was called for a statement that
selects a TIME(n) column, resulting in a loss of trailing
zeros in the returned result. Now, the MySqlTime class
calculates ticks, rather than converting the microseconds
to a string. (Bug #31623730, Bug #100218)

* MySqlConnection.GetSchema(“Procedures”) returned the
literal string System.Byte[] as the value of the
ROUTINE_DEFINITION column, rather than the actual routine
definition. (Bug #31622907, Bug #100208)

* Valid query parameters of type DateTime were
misinterpreted as a string values. (Bug #31598178, Bug
#100159)

* A mismatch of data types between the parameter of a
stored procedure and the corresponding MySqlParameter
when the Prepare() method was called did not generate an
exception. (Bug #31458774, Bug #99793)

* An SQL syntax error was reported on valid code for
creating a dynamic dropdown list from data in an ASP.NET
Core application. (Bug #31337609, Bug #99523)

* Entity Framework code-first migration omitted the schema
attribute that was assigned to an entity, although the
automatic migration appeared to generate a table with the
proper schema value at first. Subsequent queries using
the schema name returned errors. (Bug # 31323788, Bug
#94343)

* The mapping from the TINYINT and BIT data types to
BOOLEAN was not performed as expected when scaffolding
was used. (Bug #31304070, Bug #99419)

* The Entity Framework Core migration script replaced NULL
with NOT NULL on a column in the migrated table when the
MaxLength attribute of the property for it was changed in
the model. (Bug #31070175, Bug #96913)

* The Ubiety.Dns.Core.dll binary included with the
MySQL.Data NuGet package was built in debug mode, which
prevented the publishing of applications to Microsoft
Store. (Bug #31061034, Bug #98955)

* An application using Entity Framework code-first
migration without the default system decimal separator,
the period character (.), could not generate a new
database. Now, setting the system decimal separator to a
different character is permitted. (Bug #30965702, Bug
#94358)

* Every column of type CHAR(36) was interpreted as a GUID,
which could cause the first query made by a restarted
application using the MySQL.Data package to return an
exception. This fix introduces a dedicated format
(8-4-4-4-12) to interpret the column type properly. (Bug
#29963760, Bug #93399)

* Connector/NET code did not read from the
MySql.Data.Properties.ReservedWords.txt assembly resource
consistently and could return an incomplete set of
reserved keywords. (Bug #27536342, Bug #89639)

* No error occurred when MySqlCommand.CommandTimeout was
set to a negative number, however, subsequently setting
it to a positive number returned an exception. (Bug
#26574860, Bug #87316)

* The TcpClient implementation limited some of the
connection options when an external wrapper from a
Windows Forms application made the connection. This fix
enables the related external destructor to be called
without returning an error. (Bug #26427802, Bug #82810)

* An exception was returned in debug mode after a command
was canceled within a connection that was not null when
CancelQuery was called. Thanks to Denis Yarkovoy for the
improved validation patch. (Bug #26362494, Bug #86836)

* The DbContext.Database.Migrate() method did not succeed
because the __efmigrationshistory table was not found.
This fix modifies the method that validates the existence
of the table. (Bug #25901276, Bug #85902)

* When a parameter value of zero was passed to the
MySqlParameter constructor, Connector/NET used the
MySqlDbType enumeration by default for type mapping and
changed the value to NULL. This fix maps the default type
to int32. (Bug #25573071, Bug #85027)

* Entity Framework code-first migration excluded the length
specifier in the resulting binary columns. (Bug
#23171349, Bug #81179)

* Create Table statements generated with Entity Framework
were missing some of the semicolons, which caused MySQL
Server to return errors. (Bug #22669961, Bug #80159)

On Behalf of MySQL Release Engineering Team,
Surabhi Bhat


MySQL Connector/J 8.0.22 has been released

Posted on by

Dear MySQL users,

MySQL Connector/J 8.0.22 is the latest General Availability release of
the MySQL Connector/J 8.0 series. It is suitable for use with MySQL
Server versions 8.0, 5.7, and 5.6. It supports the Java Database
Connectivity (JDBC) 4.2 API, and implements the X DevAPI.

This release includes the following new features and changes, also
described in more detail on

https://dev.mysql.com/doc/relnotes/connector-j/8.0/en/news-8.0.22.html

As always, we recommend that you check the “CHANGES” file in the
download archive to be aware of changes in behavior that might affect
your application.

To download MySQL Connector/J 8.0.22 GA, see the “General Availability
(GA) Releases” tab at http://dev.mysql.com/downloads/connector/j/

Enjoy!

Changes in MySQL Connector/J 8.0.22 (2020-10-19, General
Availability)

Functionality Added or Changed


     * Security Enhancement: Previously, the LOCAL data loading
       capability for the LOAD DATA statement can be controlled on the
       client side only by enabling it for all files accessible to the
       client or by disabling it altogether, using the connection
       property allowLoadLocalInfile. A new connection property,
       allowLoadLocalInfileInPath, has been introduced to allow LOCAL
       data loading only from a specific filepath; see the description
       for the option in Configuration Properties
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-configuration-properties.html)
       for details.

     * X DevAPI: A new connection property,
       xdevapi.compression-algorithms, has been introduced for
       specifying the compression algorithms to use and the priority in
       which they will be negotiated for.  Also, the older connection
       property xdevapi.compression-algorithm (without an “s” at the end
       of its name) has been renamed to xdevapi.compression-extensions;
       its function remains the same as before (providing
       implementations for compression algorithms), but the syntax for
       its value has been changed: each element in a triplet for an
       algorithm is now separated by a colon (:). See Connection
       Compression Using X DevAPI
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-connection-compression-xdevapi.html)
       for details.

     * X DevAPI: The asynchronous variant of the X Protocol is no
       longer supported by Connector/J; the connection properties
       useAsyncProtocol and asyncResponseTimeout are now deprecated and
       have no effect when used.

     * X DevAPI: Connector/J now supports Java keystore for SSL client
       certificates for X DevAPI sessions. The following new connection
       properties have been introduced for the purpose:

          + xdevapi.ssl-keystore

          + xdevapi.ssl-keystore-type

          + xdevapi.ssl-keystore-password
       See Connecting Securely Using SSL
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-using-ssl.html)
       for details.

     * When trying to open multiple connections to a MySQL server using
       Connector/J with a named pipe on Windows systems, the attempt
       sometimes failed with the “All pipe instances are busy” error.
       With this fix, if a timeout has been set using the connection
       property connectTimeout or the method
       DriverManager.setLoginTimeout(), Connector/J will retry opening
       the named pipe repeatedly until the timeout is reached or the
       named pipe is opened successfully. As a side-effect of this new
       behavior, there will be a delay, equal to the length of the
       timeout, for throwing an error for a failed named-pipe connection
       even when it is caused by an error other than “All pipe instances
       are busy.” (Bug #31711961, Bug #98667)

     * When the connection option sslMode is set to VERIFY_IDENTITY,
       Connector/J now validates the host name in the connection string
       against the host names or IP addresses provided under the Subject
       Alternative Name (SAN) extension in the server’s X.509
       certificate. Also, verification against the Common Name (CN) is
       now performed when a SAN is not provided in the certificate or if
       it does not contain any DNS name or IP address entries. Host
       names listed in the certificate, under either the SAN or the CN,
       can contain a wildcard character as specified in the RFC 6125
       standard. Thanks to Daniël van Eeden for contributing to the
       patch. (Bug #31443178, Bug #99767, Bug #28834903, Bug #92903)

     * When using Connector/J, the AbandonedConnectionCleanupThread
       thread can now be disabled completely by setting the new system
       property com.mysql.disableAbandonedConnectionCleanup to true when
       configuring the JVM. The feature is for well-behaving
       applications that always close all connections they create.
       Thanks to Andrey Turbanov for contributing to the new feature.
       (Bug #30304764, Bug #96870)

     * Connector/J now supports client authentication with MySQL
       Enterprise Server using simple or SASL (SCRAM-SHA-1) LDAP
       authentication
(https://dev.mysql.com/doc/refman/8.0/en/ldap-pluggable-authentication.html#ldap-pluggable-authentication-usage)
       on Windows and Linux platforms.

     * Connector/J can now be prevented from falling back to the
       system-wide truststore and keystore for server and client
       identity authentication, respectively. For JDBC connections, two
       new connection properties, fallbackToSystemKeyStore and
       fallbackToSystemTrustStore, have been introduced for the control.
       While those properties are true by default (fallbacks enabled),
       setting them to false disable fallbacks. See Connecting Securely
       Using SSL
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-using-ssl.html)
       for details.  X DevAPI: Similar
       control for fallback for X DevAPI connections are provided by
       the new connection properties,
       xdevapi.fallback-to-system-keystore, and
       xdevapi.fallback-to-system-truststore.

     * The integration classes for JBoss have been removed from
       Connector/J.

Bugs Fixed


     * In a load balancing setup, if the connection parameter
       loadBalanceBlacklistTimeout was set, a server that was once
       unavailable remained in the blocklist even after a connection to
       it has been reestablished, and this affected the system’s
       performance. With this fix, the server is removed from the
       blocklist as soon as it becomes available again. (Bug #31699357,
       Bug #96309)

     * Using a PreparedStatement to store a Date into a database
       sometimes resulted in a NullPointerException. It was
       because some assignments were missing in
       ServerPreparedQueryBindValue.clone(), and this patch
       corrects the issue. (Bug #31418928, Bug #99713)

* When a client attempted to establish a JDBC connection using the server’s X Protocol port, Connector/J threw an ArrayIndexOutOfBoundsException. With this fix, Connector/J throws the proper exception for using the wrong protocol with the port and returns a proper error message. (Bug #31083755, Bug #99076)

     * LocalDate, LocalDateTime, and LocalTime values set through
       Connector/J were altered when there was a timezone difference
       between the server and the client.  This fix corrects the issue
       by handling the LocalDate, LocalTime, and LocalDateTime with no
       time zone conversion and no intermediate conversions to other
       date-time classes. Thanks to Iwao Abe for his contribution to the
       fix. (Bug #29015453, Bug #93444)

Enjoy and thanks for the support!

On behalf of the MySQL Release Team,
Nawaz Nazeer Ahamed

MySQL Shell 8.0.22 for MySQL Server 8.0 and 5.7 has been released

Posted on by

Dear MySQL users,

MySQL Shell 8.0.22 is a maintenance release of MySQL Shell 8.0 Series (a component of the MySQL Server). The MySQL Shell is provided under Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and 5.7. Please upgrade to MySQL Shell 8.0.22.

MySQL Shell is an interactive JavaScript, Python and SQL console interface, supporting development and administration for the MySQL Server. It provides APIs implemented in JavaScript and Python that enable you to work with MySQL InnoDB cluster and use MySQL as a document store.

The AdminAPI enables you to work with MySQL InnoDB cluster and InnoDB ReplicaSet, providing integrated solutions for high availability and scalability using InnoDB based MySQL databases, without requiring advanced MySQL expertise. For more information about how to configure and work with MySQL InnoDB cluster and MySQL InnoDB ReplicaSet see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document collections and perform Create, Update, Read, Delete (CRUD) operations on those collections from your favorite scripting language. For more information about how to use MySQL Shell and the MySQL Document Store support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI you can also use the latest MySQL Connectors for your language of choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the legacy MySQL protocol. Additionally, MySQL Shell provides partial compatibility with the mysql client by supporting many of the same command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.22, see the
“General Availability (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

Changes in MySQL Shell 8.0.22 (2020-10-19, General Availability)

  • AdminAPI Bugs Fixed
  • Functionality Added or Changed
  • Bugs Fixed

AdminAPI Bugs Fixed

  • Due to the changes introduced in bug#31467823, AdminAPI user messages have been updated to use the same terminology. (Bug #31798724) References: See also: Bug #31462783, Bug #31467823.
  • In 8.0.22, Group Replication deprecates the group_replication_ip_whitelist system variable in favor of group_replication_ip_allowlist. Therefore AdminAPI now includes a new option named ipAllowlist and deprecates the ipWhitelist option used with dba.createCluster(), Cluster.addInstance() and Cluster.rejoinInstance(). Regardless of the target instance version, if ipWhitelist is used MySQL Shell prints a deprecation warning. When the target instance is 8.0.22 or later Group Replication
    reports a deprecation warning, but AdminAPI avoids this by ensuring the appropriate variable is set according to the version of MySQL running on the instance. If the target instance is 8.0.22 or later and the old ipWhitelist option is used, AdminAPI updates the new group_replication_ip_allowlist system variable. On instances running MySQL 8.0.21 or earlier, AdminAPI updates the older group_replication_ip_whitelist system variable.
    This work also fixes a bug in Cluster.options(), which shows the list of all Group Replication options that are configurable by MySQL Shell. When an option is not set or does not exist (for example in a certain version) it is displayed as having a null value. However, when you passed in the all option to Cluster.options(), such variables were being excluded from the list. (Bug#31798495)
  • The dba.configureLocalInstance() and dba.configureInstance() operations could not be used against instances that were part of unmanaged replication groups. This made it impossible to create a cluster administrator account, which is required when adopting a Group Replication group into an InnoDB Cluster. Creating cluster administrator accounts is very important for the remote management of InnoDB Clusters, and to avoid having to manually create the user and required privileges on
    each instance. The fix ensures that dba.configureLocalInstance() and dba.configureInstance() can be executed against instances belonging to unmanaged
    replication groups. Because such instances are ready for InnoDB Cluster usage, a message confirms this. (Bug#31691232)
  • The dba.rebootClusterFromCompleteOutage() operation was only checking GTID_EXECUTED when validating for the instance that has the most transactions. Transactions that were received (and certified) but not executed yet were not included in that check. Also, transactions received through replication channels other than the Group Replication applier were also not being considered in that check, because the instance was probably the primary while the cluster was running. Ignoring these transactions led to data loss. Now, any known and managed
    replication channels are considered as part of the check. (Bug #31673163)
  • The dba.removeInstance() operation failed if the instance being removed was not only unreachable but also unresolvable, which could be the case when the instance
    was running in a container that takes down its own DNS record when removed.
    Instances could not be removed even if force was enabled, because of a validation that ensured that the given instance is a valid address by checking if it is
    resolvable. If that validation failed, nothing else was attempted. The fix completely removes address resolution from the whole AdminAPI. That check was redundant,
    because invalid addresses would eventually lead to an error anyway when a connection was opened. IPv6 address syntax validations were left to remind you that ::1 should be specified as [::1]. (Bug #31632606)
  • Calling Cluster.rejoinInstance(“host:port”) with no user name specified caused AdminAPI to try and connect using the operating system user name instead of the credentials used to connect the cluster object. Now, if credentials are not provided, they are taken from the target server’s connection options. (Bug #31632554)
  • When adding an instance to a cluster using MySQL Clone and monitoring the transfer of data, MySQL Shell could stop unexpectedly. This was due to the assumption that the Performance Schema would provide information on all four stages of cloning, which might not exist if the data set was very small. The fix ensures that updates are performed according to the information which is available
    at each time. (Bug #31545728)
  • When an unreachable primary instance was forcibly removed from a cluster, the group_replication_group_seeds system variable was not updated because the group status was queried from the primary, which was missing. The cluster was left in an inconsistent state, and if any of the instances were restarted, they would not be able to automatically rejoin, because group_replication_group_seeds contained an invalid
    address, which caused Group Replication to abort trying to join the group.
    (Bug #31531704)
  • If the validate_password plugin was enabled, the setupAdminAccount() and setupRouterAccount() operations would fail with an error indicating the password did not meet the policy requirements. This happened regardless of whether the password met the policy requirements or not. (Bug #31491092)
  • In the event of dba.createCluster() failing, the metadata record was left behind and Group Replication was also started. The cluster was left in an inconsistent state
    that could not be recovered from when calling dba.createCluster() again. Now, metadata changes during a dba.createCluster() operation are enclosed in a
    transaction, so that both the cluster and instance records are only committed if the operation succeeds. Group Replication is also stopped if the create operation
    does not complete successfully. Similarly, Cluster.addInstance() has been changed to ensure that its metadata record is only inserted when everything else has completed successfully. On retry, it skips any steps that would cause conflicts because Group Replication is already running. This introduces a behavior change, where adding an instance that is part of the group but not in the metadata just adds it to the metadata, instead of aborting and requiring you to execute Cluster.rescan(). (Bug #31455419)
  • If a connection timed out during a Cluster.status() operation, MySQL Shell could appear to hang for a long time. To improve the responsiveness, the default timeout
    of AdminAPI operations has been reduced from 10 seconds to 2 seconds. This ensures operations like Cluster.status() do not appear to freeze for a long time
    when there are unreachable instances. (Bug #30884174)
  • Instances operating in an InnoDB Cluster or InnoDB ReplicaSet are all required to have the same password for the administrative account. In a situation where the
    password on an instance joining a InnoDB Cluster or InnoDB ReplicaSet did not match the other instances, the error message did not explain this and the instance
    failed to join. Now, in such a situation the error is detected and the resulting message mentions that the password is the cause of the failure. It is recommended
    that you set up administrator accounts using the setupAdminAccount() operation, see Creating User Accounts for Administration (https://dev.mysql.com/doc/refman/8.0/en/admin-api-overvi ew.html#creating-user-accounts-for-admin-api).
    (Bug#30728744)

Functionality Added or Changed

  • Two new utilities are available in MySQL Shell to export single tables from a MySQL server instance.
    • MySQL Shell’s new table dump utility util.dumpTables() supports the export of a selection of tables or views from a schema, from an on-premise
      MySQL instance into an Oracle Cloud Infrastructure Object Storage bucket or a set of local files. It works in the same way as the instance dump utility
      util.dumpInstance() and schema dump utility util.dumpSchemas() introduced in 8.0.21, but with a different selection of suitable options. The exported items can then be imported into a MySQL Database Service DB System (a MySQL DB System, for short) or a MySQL Server instance using MySQL Shell’s dump loading utility util.loadDump().
    • MySQL Shell’s new table export utility util.exportTable() exports a MySQL relational table into a data file in a variety of formats, either on the local server or in an Oracle Cloud Infrastructure Object Storage bucket. The data can
      then be uploaded into a table on a target MySQL server using MySQL Shell’s parallel table import utility util.importTable(), which uses parallel connections to provide rapid data import for large data files. The data file can also be used to import data to a different application, or as a lightweight logical backup for a single data table.
  • From MySQL Shell 8.0.22, when you export schemas to an Oracle Cloud Infrastructure Object Storage bucket using MySQL Shell’s instance dump utility and schema dump utility, util.dumpInstance() and util.dumpSchemas(), during the dump you can generate a pre-authenticated request URL for every item. The utilities do this by default when the ocimds option is set to true, and you can control the feature using the ociParManifest and ociParExpireTime options. The user account that runs MySQL Shell’s dump loading utility util.loadDump() then uses the pre-authenticated request URLs to load the dump files without additional access permissions.

Bugs Fixed

  • MySQL Shell’s dump loading utility util.loadDump() would stop with an error if a data file’s size was larger than an applicable server limit relating to the maximum
    transaction size, such as the max_binlog_cache_size limit. Now, the utility stops the data load in mid-file if the number of bytes uploaded is about to exceed 1.5 times the bytesPerChunk setting of the utility that created the data files, which is stored in the dump metadata. The data load is then restarted to upload the remainder of the file. Due to this new safeguard, the default bytesPerChunk setting of MySQL Shell’s instance dump utility util.dumpInstance(), schema dump utility util.dumpSchemas(), and table dump utility util.dumpTables() has been increased from 32 MB to 64 MB.
    (Bug #31945539)
  • MySQL Shell’s instance dump utility util.dumpInstance() and schema dump utility util.dumpSchemas() did not take the row size into account when deciding whether to chunk table data, only the number of rows. A table with a smaller number of rows containing large amounts of data might therefore bypass chunking even if the row size exceeded the specified chunk size for the dump. The utilities now carry out chunking regardless of the number of rows, unless the data is estimated to fit in a single chunk of the specified size. (Bug #31938831)
  • When MySQL Shell’s instance dump utility util.dumpInstance() or schema dump utility util.dumpSchemas() was splitting a table into chunks, an integer overflow, loop, and consequent out of memory error could occur if the maximum value in the index
    column was close to the maximum value of an integer. Extra checks have now been added to avoid this situation. (Bug #31896448)
  • Corrupted SQL files could occur in MySQL Shell’s dumps if the dumped string was exactly 2048 bytes, due to an internal module error which is now accounted for.
    (Bug#31843832)
  • The administrative user account on an Oracle Cloud Infrastructure Compute instance, which was used when importing data with MySQL Shell’s dump loading utility util.loadDump(), was unable to revoke privileges on MySQL system schemas (mysql and sys) that it did not have itself. Affected REVOKE statements are now stripped from an instance or schema dump created by MySQL Shell’s instance dump utility util.dumpInstance() and schema dump utility util.dumpSchemas() when the strip_restricted_grants compatibility option is used. (Bug #31842532)
  • MySQL Shell’s instance dump utility and schema dump utility, dumpInstance() and dumpSchemas(), previously fetched the value of the gtid_executed system variable
    before the read lock was established, which could potentially lead to an inconsistency with the dumped data. The GTID set is now retrieved while the read lock is active. (Bug #31706940)
  • From MySQL Shell 8.0.22, you can use the -pym command-line option to execute a specified Python module as a script in MySQL Shell’s Python mode. –pym works in
    the same way as Python’s -m command line option. (Bug#31694202)
  • In MySQL Shell in Python mode, functions registered in JavaScript could not be called from global extension objects if they had optional arguments that were not
    provided. (Bug #31693096)
  • MySQL Shell treated the character sequence */ as the end of a comment even when it was part of a quoted string. (Bug #31689135)
  • MySQL Shell’s instance dump utility and schema dump utility, dumpInstance() and dumpSchemas(), automatically select an index column to order and chunk the data. When an index with a functional key part was present for a table, the query for index columns returned NULL for the column name, causing an exception in the utility. These column names are now filtered out in the query. (Bug#31687059)
  • MySQL Shell’s parallel table import utility util.importTable() has a new option decodeColumns, and an enhancement to the columns option, to enable you to
    capture columns from the import file for input preprocessing (or to discard them) in the same way as with a LOAD DATA statement. The decodeColumns option
    specifies preprocessing transformations for the captured data in the same way as the SET clause of a LOAD DATA statement, and assigns them to columns in the target table. (Bug #31683641)
  • MySQL Shell’s dump loading utility loadDump() now verifies that it can open and write to the progress state file before it starts to retrieve the dump files, so that
    the utility does not spend time fetching the dump files if the import is subsequently going to fail for that reason. (Bug #31667539)
  • Before MySQL 8.0, user accounts that have all privileges used the statement GRANT ALL PRIVILEGES, rather than the full list of privileges. When MySQL Shell’s instance dump utility dumpInstance() was used to dump an instance, ALL PRIVILEGES was stripped from the statement, leaving the accounts with no privileges. The utility now replaces this grant with the administrator role.
    (Bug #31661180)
  • MySQL Shell’s instance dump utility and schema dump utility, dumpInstance() and dumpSchemas(), and dump loading utility loadDump(), previously timed out after
    the main thread had been idle for 8 hours. The timeout has now been extended indefinitely (to 1 year) so that long data load times do not cause the dump or import to fail. (Bug #31652265)
  • MySQL Shell’s dump loading utility loadDump() now includes a comment in its LOAD DATA statements to identify the data chunk that is currently being loaded.
    If you need to cancel the import, you can use this information to decide whether to let the import of this chunk complete or stop it immediately. (Bug #31646650)
  • Previously, MySQL Shell’s dump loading utility loadDump() closed DDL files after all tables from the schema had finished loading. For a dump with a very large number of DDL files, this could lead to unexplained runtime errors being returned. The utility now closes DDL files immediately after reading them. (Bug #31645896)
  • MySQL Shell’s dump loading utility loadDump() previously used only its main thread to execute the DDL scripts for tables. For a dump containing a large number of tables, fetching the DDL scripts could have a significant impact on the time taken. The utility now fetches and executes DDL scripts for tables using all its threads, with the exception of DDL scripts for views, which are fetched in parallel but executed only in the main thread to avoid race conditions. (Bug #31645806)
  • MySQL Shell’s dump loading utility loadDump() now loads views in sequence and only after all tables and placeholders from all schemas have been loaded, ensuring
    that views do not reference items that do not yet exist. (Bug #31645792)
  • MySQL Shell now ignores any other available Python installations when a bundled compatible version of the Python interpreter has been installed. (Bug #31642521)
  • When MySQL Shell’s dump loading utility loadDump() is importing users and their roles and grants, an error is now returned if the user already exists in the target
    instance, and the user’s grants from the dump files are not applied. Previously, the grants were applied to the existing user. (Bug #31627432)
  • MySQL Shell’s dump loading utility loadDump() now has an option updateGtidSet to apply the gtid_executed GTID set from the source MySQL instance to the gtid_purged GTID set on the target MySQL instance. You can append or replace the GTID set depending on the release of the target MySQL instance. (Bug #31627419)
  • MySQL Shell’s instance dump utility dumpInstance() and dump loading utility loadDump() now have options to include (includeUsers) or exclude (excludeUsers ) named user accounts from the dump files or from the import. You can use these options to exclude user accounts that are not accepted for import to a MySQL DB System, or that already exist or are not wanted on the target MySQL instance.
    (Bug #31627292)
  • With the deferTableIndexes option set to all, MySQL Shell’s dump loading utility loadDump() defers creation of all secondary indexes until after the table is loaded.
    Previously, a table with a unique key column containing an auto-increment value failed to load in this situation. The utility now also creates indexes defined on columns with auto-increment values when the deferTableIndexes option is set to all. (Bug #31602690)
  • The upload method used by MySQL Shell’s instance dump utility util.dumpInstance() and schema dump utility util.dumpSchemas() to transfer files to an Oracle Cloud
    Infrastructure Object Storage bucket has a file size limit of 1.2 TiB. In MySQL Shell 8.0.21, the multipart size setting means that the numeric limit on multiple file parts applies first, creating a limit of approximately 640 GB. From MySQL Shell 8.0.22, the
    multipart size setting has been changed to allow the full file size limit.
    (Bug #31589858)
  • MySQL Shell’s upgrade checker utility checkForServerUpgrade() now checks for the obsolete NO_AUTO_CREATE_USER SQL mode. (Bug #31501981, Bug #99903)
  • The parallelization of table loading by MySQL Shell’s dump loading utility loadDump() has been improved. (Bug#31441903)
  • If the settings for the server’s global character set variables differed from the settings for the current session, MySQL Shell’s parallel table import utility importTable() could not import data into a table created in the session whose name contained non-ASCII characters. Now, when the utility’s characterSet option is specified, MySQL Shell executes a SET NAMES statement with the given value. (Bug #31412330)
  • MySQL Shell’s parallel table import utility importTable() could not import data if the global SQL mode NO_BACKSLASH_ESCAPES was set. The utility now clears the
    global SQL mode in sessions created to run the import. (Bug #31407133)
  • When a function that was defined as a member of a MySQL Shell extension object had a number of optional parameters but no required parameters, in some situations
    calling the function with zero parameters or one parameter returned an error.
    (Bug #30744994)
  • MySQL Shell’s db global object did not return the current schema when its properties were queried. (Bug #30296825, Bug #96839)

On Behalf of Oracle/MySQL Release Engineering Team,
Prashant Tekriwal


MySQL Shell 8.0.21 for MySQL Server 8.0 and 5.7 has been released

Posted on by

Dear MySQL users,

MySQL Shell 8.0.21 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.21.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster and InnoDB
ReplicaSet, providing integrated solutions for high availability and scalability
using InnoDB based MySQL databases, without requiring advanced MySQL
expertise.  For more information about how to configure and work with
MySQL InnoDB cluster and MySQL InnoDB ReplicaSet see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.  For more
information about how to use MySQL Shell and the MySQL Document Store
support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.21, see the
“General Availability (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

Changes in MySQL Shell 8.0.21 (2020-07-13, General Availability)

     * AdminAPI Added or Changed Functionality
     * AdminAPI Bugs Fixed
     * Functionality Added or Changed
     * Bugs Fixed

AdminAPI Added or Changed Functionality

     * A new user configurable tag framework has been added to
       the metadata, to allow specific instances of a cluster or
       ReplicaSet to be marked with additional information. Tags
       can be any ASCII character and provide a namespace. You
       set tags for an instance using the setInstanceOption()
       operation. In addition, AdminAPI and MySQL Router 8.0.21
       support specific tags, which enable you to mark instances
       as hidden and remove them from routing. MySQL Router then
       excludes such tagged instances from the routing
       destination candidates list. This enables you to safely
       take a server instance offline, so that applications and
       MySQL Router ignore it, for example while you perform
       maintenance tasks, such as server upgrade or
       configuration changes. To bring the instance back online,
       use the setInstanceOption() operation to remove the tags
       and MySQL Router adds the instance back to the routing
       destination candidates list, and it becomes online for
       applications. For more information, see Tagging the
       Metadata
(https://dev.mysql.com/doc/refman/8.0/en/admin-api-tagging.html).

AdminAPI Bugs Fixed

     * Important Change: Previously, Group Replication did not
       support binary log checksums, and therefore one of the
       requirements for instances in InnoDB cluster was that
       binary log checksums were disabled by having the
       binlog_checksum system variable set to NONE. AdminAPI
       verified the value of binlog_checksum during the
       dba.checkInstanceConfiguration() operation and disallowed
       creating a cluster or adding an instance to a cluster
       that did not have binary log checksums disabled. In
       version 8.0.21, Group Replication has lifted this
       restriction, therefore InnoDB cluster now permits
       instances to use binary log checksums, with
       binlog_checksum set to CRC32. The setting for
       binlog_checksum does not have to be the same for all
       instances. In addition, sandboxes deployed with version
       8.0.21 and later do not set the binlog_checksum variable,
       which defaults to CRC32. (Bug #31329024)

     * Adopting a Group Replication setup as a cluster can be
       performed when connected to any member of the group,
       regardless of whether it is a primary or a secondary.
       However, when a secondary member was used,
       super_read_only was being incorrectly disabled on that
       instance. Now, all operations performed during an
       adoption are done using the primary member of the group.
       This ensures that no GTID inconsistencies occur and that
       super_read_only is not incorrectly disabled on secondary
       members. (Bug #31238233)

     * Using the clusterAdmin option to create a user which had
       a netmask as part of the host resulted in an error when
       this user was passed to the dba.createCluster()
       operation. Now, accounts that specify a netmask are
       treated as accounts with wildcards, meaning that further
       checks to verify if the account accepts remote
       connections from all instances are skipped. (Bug
       #31018091)

     * The check for instance read-only compatibility was using
       a wrong MySQL version as the base version. The
       cross-version policies were added to Group Replication in
       version 8.0.17, but the check was considering instances
       running 8.0.16. This resulted in a misleading warning
       message indicating that the added instance was read-only
       compatible with the cluster, when this was not true (only
       for instances 8.0.16). The fix ensures that the check to
       verify if an instance is read-compatible or not with a
       cluster is only performed if the target instance is
       running version 8.0.17 or later. (Bug #30896344)

     * The maximum number of instances in an InnoDB cluster is
       9, but AdminAPI was not preventing you from trying to add
       more instances to a cluster and the resulting error
       message was not clear. Now, if a cluster has 9 instances,
       Cluster.addInstance prevents you adding more instances.
       (Bug #30885157)

     * Adding an instance with a compatible GTID set to a InnoDB
       cluster or InnoDB ReplicaSet on which provisioning is
       required should not require any interaction, because this
       is considered a safe operation. Previously, in such a
       scenario, when MySQL Clone was supported MySQL Shell
       still prompted to choose between cloning or aborting the
       operation. Now, the operation proceeds with cloning,
       because this is the only way to provision the instance.
       Note
       instances with an empty GTID set are not considered to
       have a compatible GTID set when compared with the InnoDB
       cluster or InnoDB ReplicaSet. Such scenarios are
       considered to be unknown, therefore MySQL Shell prompts
       to confirm which action should be taken.
       (Bug #30884590)

     * The Group Replication system variables (prefixed with
       group_replication) do not exist if the plugin has not
       been loaded. Even if the system variables are persisted
       to the instance’s option file, they are not loaded unless
       the Group Replication plugin is also loaded when the
       server starts. If the Group Replication plugin is
       installed after the server starts, the option file is not
       reloaded, so all system variables have default values.
       Instances running MySQL 8.0 do not have a problem because
       SET PERSIST is used. However, on instances running
       version MySQL 5.7, the dba.rebootCluster() operation
       could not restore some system variables if the Group
       Replication plugin was uninstalled. Now, the
       dba.configureInstance() operation persists the Group
       Replication system variables to configuration files with
       the loose_ prefix. As a result, once the Group
       Replication plugin is installed, on instances running 5.7
       the persisted values are used instead of the default
       values. (Bug #30768504)

     * The updateTopologyMode option has been deprecated and the
       behavior of Cluster.rescan() has been changed to always
       update the topology mode in the Metadata when a change is
       detected. MySQL Shell now displays a message whenever
       such a change is detected. (Bug #29330769)

     * The cluster.addInstance() and cluster.rejoinInstance()
       operations were not checking for the full range of
       settings which are required for an instance to be valid
       for adding to the cluster. This resulted in attempts to
       use instances which run on different operating systems to
       fail. For example, a cluster running on two instances
       that were hosted on a Linux based operating system would
       block the addition of an instance running Microsoft
       Windows. Now, cluster.addInstance() and
       cluster.rejoinInstance() operations validate the instance
       and prevent adding or rejoining an instance to the
       cluster if the value of the lower_case_table_names,
       group_replication_gtid_assignment_block_size or
       default_table_encryption of the instance are different
       from the ones on the cluster. (Bug #29255212)

Functionality Added or Changed

     * MySQL Shell now has an instance dump utility,
       dumpInstance(), and schema dump utility, dumpSchemas().
       The new utilities support the export of all schemas or a
       selected schema from an on-premise MySQL server instance
       into an Oracle Cloud Infrastructure Object Storage bucket
       or a set of local files. The schemas can then be imported
       into a MySQL Database Service DB System using MySQL
       Shell’s new dump loading utility. The new utilities
       provide Oracle Cloud Infrastructure Object Storage
       streaming, MySQL Database Service compatibility checks
       and modifications, parallel dumping with multiple
       threads, and file compression.

     * MySQL Shell’s new dump loading utility, loadDump(),
       supports the import of schemas dumped using MySQL Shell’s
       new instance dump utility and schema dump utility into a
       MySQL Database Service DB System. The dump loading
       utility provides data streaming from remote storage,
       parallel loading of tables or table chunks, progress
       state tracking, resume and reset capability, and the
       option of concurrent loading while the dump is taking
       place.

     * The X DevAPI implementation now supports JSON schema
       validation, which enables you to ensure that your
       documents have a certain structure before they can be
       inserted or updated in a collection. To enable or modify
       JSON schema validation you pass in a JSON object like:
{
    validation: {
      level: “off|strict”,
      schema: “json-schema”
    }
}

       Here, validation is JSON object which contains the keys
       you can use to configure JSON schema validation. The
       first key is level, which can take the value strict or
       off. The second key, schema, is a JSON schema, as defined
       at http://json-schema.org. If the level key is set to
       strict, documents are validated against the json-schema
       when they are added to the collection, or when an
       operation updates the document. If the document does not
       validate, the server generates an error and the operation
       fails. If the level key is set to off, documents are not
       validated against the json-schema.
       You can pass a validation JSON object to the
       schema.createCollection() operation, to enable JSON
       schema validation, and schema.modifyCollection()
       operation, to change the current JSON schema validation,
       for example to disable validation. For more information,
       see JSON Schema Validation
(https://dev.mysql.com/doc/x-devapi-userguide/en/collection-validation.html).

Bugs Fixed

     * MySQL Shell plugins now support the use of the **kwargs
       syntax in functions defined in Python that are made
       available by the plugin. Using **kwargs in a function
       definition lets you call the function using a
       variable-length list of keyword arguments with arbitrary
       names. If the function is called from MySQL Shell’s
       JavaScript mode, MySQL Shell passes the named arguments
       and their values into a dictionary object for the Python
       function. MySQL Shell first tries to associate a keyword
       argument passed to a function with any corresponding
       keyword parameter that the function defines, and if there
       is none, the keyword argument is automatically included
       in the **kwargs list. As a side effect of this support,
       any API function called from Python in MySQL Shell that
       has a dictionary of options as the last parameter
       supports defining these options using named arguments.
       (Bug #31495448)

     * When switching to SQL mode, MySQL Shell queries the SQL
       mode of the connected server to establish whether the
       ANSI_QUOTES mode is enabled. Previously, MySQL Shell
       could not proceed if it did not receive a result set in
       response to the query. The absence of a result is now
       handled appropriately. (Bug #31418783, Bug #99728)

     * In SQL mode, when the results of a query are to be
       printed in table format, MySQL Shell buffers the result
       set before printing, in order to identify the correct
       column widths for the table. With very large result sets,
       it was possible for this practice to cause an out of
       memory error. MySQL Shell now buffers a maximum of 1000
       rows for a result set before proceeding to format and
       print the table. Note that if a field in a row after the
       first 1000 rows contains a longer value than previously
       seen in that column in the result set, the table
       formatting will be misaligned for that row. (Bug
       #31304711)

     * Context switching in MySQL Shell’s SQL mode has been
       refactored and simplified to remove SQL syntax errors
       that could be returned when running script files using
       the source command. (Bug #31175790, Bug #31197312, Bug
       #99303)

     * The user account that is used to run MySQL Shell’s
       upgrade checker utility checkForServerUpgrade()
       previously required ALL privileges. The user account now
       requires only the RELOAD, PROCESS, and SELECT privileges.
       (Bug #31085098)

     * In Python mode, MySQL Shell did not handle invalid UTF-8
       sequences in strings returned by queries. (Bug #31083617)

     * MySQL Shell’s parallel table import utility importTable()
       has a new option characterSet, which specifies a
       character set encoding with which the input data file is
       interpreted during the import. Setting the option to
       binary means that no conversion is done during the
       import. When you omit this option, the import uses the
       character set specified by the character_set_database
       system variable to interpret the input data file. (Bug
       #31057707)

     * On Windows, if the MySQL Shell package was extracted to
       and used from a directory whose name contained multi-byte
       characters, MySQL Shell was unable to start. MySQL Shell
       now handles directory names with multi-byte characters
       correctly, including when setting up Python, loading
       prompt themes, and accessing credential helpers. (Bug
       #31056783)

     * MySQL Shell’s JSON import utility importJSON() now
       handles UTF-8 encoded files that include a BOM (byte mark
       order) at the start, which is the sequence 0xEF 0xBB
       0xBF. As a workaround in earlier releases, remove this
       byte sequence, which is not needed. (Bug #30993547, Bug
       #98836)

     * When the output format was set to JSON, MySQL Shell’s
       upgrade checker utility checkForServerUpgrade() included
       a description and documentation link for a check even if
       no issues were found. These are now omitted from the
       output, as they are with the text output format. (Bug
       #30950035)

On Behalf of Oracle/MySQL Release Engineering Team,
Sreedhar S

MySQL Shell 8.0.20 for MySQL Server 8.0 and 5.7 has been released

Posted on by

Dear MySQL users,

MySQL Shell 8.0.20 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.20.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster and InnoDB
ReplicaSet, providing integrated solutions for high availability and scalability
using InnoDB based MySQL databases, without requiring advanced MySQL
expertise.  For more information about how to configure and work with
MySQL InnoDB cluster and MySQL InnoDB ReplicaSet see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.  For more
information about how to use MySQL Shell and the MySQL Document Store
support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.20, see the
“General Availability (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

Changes in MySQL Shell 8.0.20 (2020-04-27, General Availability)

     * AdminAPI Added or Changed Functionality

     * AdminAPI Bugs Fixed

     * Functionality Added or Changed

     * Bugs Fixed

AdminAPI Added or Changed Functionality

     * MySQL Shell now enables you to create and configure the
       MySQL user accounts required by InnoDB cluster, InnoDB
       ReplicaSet, and MySQL Router using AdminAPI operations.
       Previously, accounts required by InnoDB cluster and
       InnoDB ReplicaSet had to be configured using the
       clusterAdmin option, and accounts required by MySQL
       Router had to be configured manually using SQL. The
       following AdminAPI operations are now available:

          + use Cluster.setupAdminAccount(user, [options]) and
            Replicaset.setupAdminAccount(user, [options]) to
            configure a MySQL user account with the necessary
            privileges to administer an InnoDB cluster or InnoDB
            ReplicaSet.

          + use Cluster.setupRouterAccount(user, [options]) and
            Replicaset.setupRouterAccount(user, [options]) to
            create a MySQL user account or upgrade an existing
            account so that it that can be used by MySQL Router
            to operate on an InnoDB cluster or InnoDB
            ReplicaSet. This is now the recommended method of
            adding MySQL Router accounts to use with InnoDB
            cluster and InnoDB ReplicaSet.

     * AdminAPI now uses a locking mechanism to avoid different
       operations from performing changes on an InnoDB
       ReplicaSet simultaneously. Previously, different
       instances of MySQL Shell could connect to an InnoDB
       ReplicaSet at the same time and execute AdminAPI
       operations simultaneously. This could lead to
       inconsistent instance states and errors, for example if
       ReplicaSet.addInstance() and ReplicaSet.setPrimary() were
       executed in parallel.
       Now, the InnoDB ReplicaSet operations have the following
       locking:

          + dba.upgradeMetadata() and dba.createReplicaSet() are
            globally exclusive operations. This means that if
            MySQL Shell executes these operations on an InnoDB
            ReplicaSet, no other operations can be executed
            against the InnoDB ReplicaSet or any of its
            instances.

          + ReplicaSet.forcePrimaryInstance() and
            ReplicaSet.setPrimaryInstance() are operations that
            change the primary. This means that if MySQL Shell
            executes these operations against an InnoDB
            ReplicaSet, no other operations which change the
            primary, or instance change operations can be
            executed until the first operation completes.

          + ReplicaSet.addInstance(),
            ReplicaSet.rejoinInstance(), and
            ReplicaSet.removeInstance() are operations that
            change an instance. This means that if MySQL Shell
            executes these operations on an instance, the
            instance is locked for any further instance change
            operations. However, this lock is only at the
            instance level and multiple instances in an InnoDB
            ReplicaSet can each execute one of this type of
            operation simultaneously. In other words, at most
            one instance change operation can be executed at a
            time, per instance in the InnoDB ReplicaSet.

          + dba.getReplicaSet() and ReplicaSet.status() are
            InnoDB ReplicaSet read operations and do not require
            any locking.
       References: See also: Bug #30349849.

     * Use the –replicaset option to configure MySQL Shell to
       work with an InnoDB ReplicaSet at start up. You must
       specify a connection to a replica set instance for this
       option to work correctly. If a replica set is found, this
       option populates the rs global object, which can then be
       used to work with the InnoDB ReplicaSet. As part of this
       addition, the –redirect-primary and –redirect-secondary
       options have been updated to also work with InnoDB
       ReplicaSet.
       When running MySQL Shell, use the
       shell.connectToPrimary([connectionData, password]) to
       check whether the target instance belongs to an InnoDB
       cluster or InnoDB ReplicaSet. If so, MySQL Shell opens a
       new session to the primary, sets the global session to
       the established session and returns it. If no
       connectionData is provided, the current global session is
       used.

AdminAPI Bugs Fixed

     * During distributed recovery which is using MySQL Clone,
       the instance restarts after the data files are cloned,
       but if the instance has to apply a large back log of
       transactions to finish the recovery process, then the
       restart process could take longer than the default 1
       minute timeout. Now, when there is a large back log use
       the dba.restartWaitTimeout option to configure a longer
       timeout to ensure the apply process has time to process
       the transactions. (Bug #30866632)

     * The dba.deleteSandboxInstance() operation did not provide
       an error if you attempted to delete a sandbox which did
       not exist. Now, in such a situation the
       dba.deleteSandboxInstance() operation throws a
       runtimeError. (Bug #30863587)

     * The Cluster.forceQuorumUsingPartitionOf() operation was
       not stopping Group Replication on any reachable instances
       that were not part of the visible membership of the
       target instance, which could lead to undefined behavior
       if any of those instances were automatically rejoining
       the cluster. The fix stops Group Replication on any
       reachable instances that are not included in the new
       forced quorum membership. (Bug #30739252)

     * It was possible for AdminAPI to select an invalidated
       instance as the latest primary, despite it having a lower
       view_id. This was because the process of getting the
       primary of the InnoDB ReplicaSet was incorrectly
       reconnecting to an invalidated member if it was the last
       instance in the InnoDB ReplicaSet. (Bug #30735124)

     * When a cluster that was created with a MySQL Shell
       version lower than 8.0.19 was offline (for example after
       a server upgrade of the instances), if you then used
       MySQL Shell 8.0.19 to connect to the cluster,
       dba.upgradeMetadata() and
       dba.rebootClusterFromCompleteOutage() blocked each other.
       You could not run dba.upgradeMetadata() because it
       requires dba.rebootClusterFromCompleteOutage() to be run
       first to bring the cluster back online. And you could not
       run dba.rebootClusterFromCompleteOutage() because
       dba.upgradeMetadata() had not been run. To avoid this
       problem please upgrade to MySQL Shell 8.0.20, where the
       preconditions for dba.rebootClusterFromCompleteOutage()
       and dba.forceQuorumUsingPartitionOf() have been updated
       to ensure they are compatible with clusters created using
       earlier versions. In other words, they are available even
       if the metadata was created using an older MySQL Shell
       version. (Bug #30661129)

     * Using MySQL Clone as the distributed recovery method to
       add an instance to an InnoDB ReplicaSet resulted in a
       segmentation fault if the target instance did not support
       RESTART. Now, the ReplicaSet.addInstance() operation
       aborts in such a situation and reverts changes after the
       connection timeout limit is reached. This is because the
       add operation needs to connect to the target instance to
       finish the operation. In such a situation, if it is not
       possible to upgrade the instance to a version of MySQL
       which supports RESTART, you have to restart the server
       manually, and then issue ReplicaSet.addInstance() again
       to retry. The retry can then use incremental recovery,
       which does not trigger the clone and the subsequent
       restart. (Bug #30657911)

     * Cluster.addInstance() normally fails if there are errant
       GTIDs in the added instance, but if MySQL Clone is being
       used for distributed recovery, that check is bypassed
       because the cloning process fixes the problem. However,
       if all members are using IPv6, MySQL Clone cannot be
       used, so incremental recovery is used instead. In such a
       situation, the instance was being added to the cluster
       without errors, but the errant transaction persisted.
       Now, if all of the cluster’s online instances are using
       IPv6 addresses and the operation tries to use MySQL Clone
       for distributed recovery, an error is thrown. (Bug
       #30645697)

     * When an InnoDB cluster or InnoDB ReplicaSet is using the
       MySQL Clone plugin, AdminAPI ensures the
       performance_schema.clone_status table is cleared out when
       the clone process starts. However, in some rare and very
       specific scenarios a race condition could happen and the
       clone operation was considered to be running before
       actually clearing out the table. In this situation, the
       MySQL Shell clone monitoring could result in an
       unexpected halt.
       As part of this fix, a potential infinite loop in the
       clone monitoring phase that could happen very rarely when
       the cloning process was extremely fast has also been
       fixed. (Bug #30645665)

     * Group Replication system variable queries were being
       executed early, without considering whether the Group
       Replication plugin was installed yet. Now, the reboot
       operation has been fixed so that if system variable
       queries fail with ER_UNKNOWN_SYSTEM_VARIABLE then the
       Group Replication plugin is installed automatically. (Bug
       #30531848)

     * The Cluster.removeInstance(instance) operation was not
       correctly handling the following cases:

          + if the instance had report_host set to a different
            value from the instance_name in the metadata,
            specifying the instance using its IP failed.

          + if the instance was unreachable, it was not possible
            to remove it if the given address did not match the
            address in the metadata.

          + when an instance was OFFLINE but reachable (for
            example because Group Replication stopped but the
            server was still running),
            Cluster.removeInstance(instance) failed. Now, in
            such a situation, if you are sure it is safe to
            remove the instance, use the force=true option,
            which means that synchronization is no longer
            attempted as part of the remove operation.

          + if the instance was OFFLINE but reachable, removing
            the instance through an address that did not match
            what was in the metadata would make the operation
            appear to succeed but the instance was not actually
            removed from the metadata.
       (Bug #30501628, Bug #30625424)

     * After operations such as removing an instance or
       dissolving a cluster, the group_replication_recovery and
       group_replication_applier replication channels were not
       being removed. (Bug #29922719, Bug #30878446)

     * The default location of the MySQL option file, for
       example /etc/my.cnf, stopped being detected by the
       dba.configureInstance() operation on some platforms
       (Debian and so on). This was a regression. The fix
       ensures that the predefined paths to option files matches
       the defaults, such as /etc/my.cnf and /etc/mysql/my.cnf.
       (Bug #96490, Bug #30171324)

Functionality Added or Changed

     * A new method shell.openSession is provided in the shell
       global object to let you create and return a session
       object, rather than set it as the global session for
       MySQL Shell.

     * You can now request compression for MySQL Shell
       connections that use X Protocol, as well as those that
       use classic MySQL protocol. For X Protocol connections,
       the default is that compression is requested, and
       uncompressed connections are allowed if the negotiations
       for a compressed connection do not succeed. For classic
       MySQL protocol connections, the default is that
       compression is disabled. After the connection has been
       made, the MySQL Shell \status command shows whether or
       not compression is in use for a session.
       New compression controls in MySQL Shell let you specify
       in the connection parameters whether compression is
       required, preferred, or disabled, select compression
       algorithms for the connection, and specify a numeric
       compression level for the algorithms.

Bugs Fixed

     * When you create an extension object for MySQL Shell, the
       options key is no longer required when you specify a
       parameter of the data type “dictionary”. If you do define
       options for a dictionary, MySQL Shell validates the
       options specified by the end user and raises an error if
       an option is passed to the function that is not in this
       list. If you create a dictionary with no list of options,
       any options that the end user specifies for the
       dictionary are passed directly through to the function by
       MySQL Shell with no validation. (Bug #30986260)

     * A bug in MySQL Shell 8.0.19, affecting classic MySQL
       protocol connections only, meant that access was denied
       if a user had stored the connection’s password with MySQL
       Shell and afterwards changed it. The password store now
       removes invalid passwords and presents the user with a
       password prompt as expected. (Bug #30912984, Bug #98503)

     * When MySQL Shell’s \source command was used in
       interactive mode to execute code from a script file,
       multi-line SQL statements in the script file could cause
       MySQL Shell to enter a loop of repeatedly executing the
       script. The issue has now been fixed. (Bug #30906751, Bug
       #98625)

     * If a stored procedure was called in MySQL Shell but its
       result was not used, any subsequent SQL statement
       returned a result set error, and exiting MySQL Shell at
       that point resulted in an incorrect shutdown. MySQL Shell
       cleared the first result set retrieved by a stored
       procedure in order to run a subsequent SQL statement, but
       did not check for any additional result sets that had
       been retrieved, which were left behind and caused the
       error. This check is now carried out and the additional
       result sets are discarded before another statement is
       executed. (Bug #30825330)

     * Due to a regression in MySQL Shell 8.0.19, the upgrade
       checker utility checkForServerUpgrade() did not accept
       any runtime options if connection data was not provided
       as the first argument. The issue has been fixed and the
       utility’s argument checking has been enhanced. (Bug
       #30689606)

     * MySQL Shell, which now bundles Python 3.7.4, could not be
       built from source with Python 3.8. The incompatibilities
       have now been corrected so Python 3.8 may be used. (Bug
       #30640012)

     * MySQL Shell’s upgrade checker utility
       checkForServerUpgrade() did not flag removed system
       variables that were specified using hyphens rather than
       underscores. The utility also now continues with its
       sequence of checks if a permissions check cannot be
       performed at the required time. (Bug #30615030, Bug
       #97855)

     * MySQL Shell’s \status command showed that a connection
       was compressed if the connection had been created while
       starting MySQL Shell, but not if it was created after
       starting MySQL Shell. Compression is now shown in both
       cases. (Bug #29006903)


On Behalf of Oracle/MySQL Release Engineering Team,
Sreedhar S