MySQL Shell 8.0.18 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.18 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.18.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster, providing an
integrated solution for high availability and scalability using InnoDB
based MySQL databases, without requiring advanced MySQL expertise. For
more information about how to configure and work with MySQL InnoDB
cluster see

  https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.  For more
information about how to use MySQL Shell and the MySQL Document Store
support see

  https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

  https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

  https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

  https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

  https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

  https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.18, see the
“General Availability (GA) Releases” tab at

  http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

  http://bugs.mysql.com/

Enjoy and thanks for the support!


Changes in MySQL Shell 8.0.18 (2019-10-14, General Availability)

     * InnoDB Cluster Added or Changed Functionality

     * InnoDB Cluster Bugs Fixed

     * Functionality Added or Changed

     * Bugs Fixed

InnoDB Cluster Added or Changed Functionality


     * MySQL Shell can now optionally log SQL statements that
       are executed by AdminAPI operations, and output them to
       the console if the –verbose option is set. The
       dba.logSql MySQL Shell configuration option or
       –dba-log-sql command line option activates logging for
       these statements. Statements executed by sandbox
       operations are excluded. Viewing the statements lets you
       observe the progress of the AdminAPI operations in terms
       of SQL execution, which can help with problem diagnosis
       for any errors.

     * AdminAPI now supports IPv6 addresses if the target MySQL
       Server version is higher than 8.0.13. When using MySQL
       Shell 8.0.18 or higher, if all cluster instances are
       running 8.0.14 or higher then you can use an IPv6 or
       hostname that resolves to an IPv6 address for instance
       connection strings and with options such as localAddress,
       groupSeeds and ipWhitelist. For more information on using
       IPv6 see Support For IPv6 And For Mixed IPv6 And IPv4
       Groups
(https://dev.mysql.com/doc/refman/8.0/en/group-replication-ipv6.html).
       References: See also: Bug #29557250, Bug #30111022, Bug
       #28982989.

     * You can now reset the passwords for the internal recovery
       accounts created by InnoDB cluster, for example to follow
       a custom password lifetime policy. Use the
       Cluster.resetRecoveryAccountsPassword() operation to
       reset the passwords for all internal recovery accounts
       used by the cluster. The operation sets a new random
       password for the internal recovery account on each
       instance which is ONLINE. If an instance cannot be
       reached, the operation fails. You can use the force
       option to ignore such instances, but this is not
       recommended, and it is safer to bring the instance back
       online before using this operation. This operation only
       applies to the passwords created by InnoDB cluster and
       cannot be used to update manually created passwords.
       Note
       The user which executes this operation must have all the
       required clusterAdmin privileges, in particular CREATE
       USER, in order to ensure that the password of recovery
       accounts can be changed regardless of the password
       verification-required policy. In other words, independent
       of whether the password_require_current system variable
       is enabled or not.

     * MySQL Shell now supports specifying TLS version 1.3 and
       TLS cipher suites for classic MySQL protocol connections.
       You can use:

          + the –tls-version command option to specify TLS
            version 1.3.

          + the –tls-ciphersuites command option to specify
            cipher suites.

          + the tls-versions and tls-ciphersuites connection
            parameters as part of a URI-type connection string.
       Note
       tls-versions (plural) does not have a key-value
       equivalent, it is only supported in URI-type connection
       strings. Use tls-version to specify TLSv1.3 in a
       key-value connection string.
       To use TLS version 1.3, both MySQL Shell and MySQL server
       must have been compiled with OpenSSL 1.1.1 or higher. For
       more information see Using Encrypted Connections
(https://dev.mysql.com/doc/refman/8.0/en/encrypted-connections.html).


InnoDB Cluster Bugs Fixed


     * The Cluster.rejoinInstance() operation was not setting
       the auto increment values defined for InnoDB cluster,
       leading to the use of the default Group Replication
       behavior if the instance configuration was not properly
       persisted, for example on 5.7 servers. The fix ensures
       that the Cluster.rejoinInstance() operation updates the
       auto increment settings of the target instance. (Bug
       #30174191)

     * The output of Cluster.status() now includes the
       replicationLag field. The value is displayed in HH:MM:SS
       format and shows the time difference between the last
       transaction commit timestamp and the last transaction
       applied timestamp. This enables you to monitor the amount
       of time between the most recent transaction being
       committed and being applied on an instance. (Bug
       #30003034)

     * Cluster.addInstance() did not ensure that the MySQL Clone
       plugin was installed or loaded on all cluster instances,
       when available and not disabled. This meant that whenever
       a cluster was created using an older MySQL Shell version,
       on a target MySQL instance supporting clone, the instance
       would not have the clone plugin installed. The result was
       that any Cluster.addInstance() call that used clone would
       fail. The same issue happened if an instance was added to
       a cluster consisting of one instance using the
       incremental recovery type and afterwards the seed
       instance was removed. This resulted in all cluster
       instances not having the clone plugin installed and
       therefore any instance added using the clone recovery
       method would fail. The fix ensures that the clone plugin
       is installed on all cluster members (if available and not
       disabled) at cluster creation time and also whenever an
       instance is added to a cluster. (Bug #29954085)

     * The Cluster.rejoinInstance() operation was not checking
       the GTID consistency of an instance being rejoined to a
       cluster, which could result in data diverging. Now, the
       GTID consistency checks conducted as part of the
       Cluster.rejoinInstance() operation have been improved to
       check for irrecoverable or diverged data-sets and also
       for empty GTID sets. If an instance is found to not be
       consistent with the cluster, it is not rejoined and the
       operation fails with a descriptive error. You are also
       shown the list of errant transactions, possible outcomes
       and solutions. (Bug #29953812)

     * Cluster.describe() was retrieving information about the
       cluster’s topology and the MySQL version installed on
       instances directly from the current session. Now, the
       information is retrieved from the Metadata schema, and
       the MySQL version is not included in the information
       output by Cluster.describe(). (Bug #29648806)

     * Using a password containing the ‘ character caused
       dba.deploySandbox() to fail. Now, all sensitive data is
       correctly wrapped to avoid such issues. (Bug #29637581)

     * The Cluster.addInstance() operation creates internal
       recovery users which are required by the Group
       Replication recovery process. If the
       Cluster.addInstance() operation failed, for example
       because Group Replication could not start, the created
       recovery users were not removed. Now, in the event of a
       failure any internal users are removed. (Bug #25503159)

     * When a cluster had lost quorum and the majority of the
       cluster instances were offline except the primary, after
       reestablishing quorum and adding a new instance to the
       cluster, it was not possible to remove and add the
       previous primary instance to the cluster. This was
       because the operation failed when trying to contact
       offline instances, which was because the feature to
       verify if a Group Replication protocol upgrade is
       required was not considering the possibility of some
       cluster instances being offline (not reachable). The fix
       improves the Group Replication protocol upgrade handling
       for the Cluster.removeInstance() operation, which now
       attempts to connect to other cluster instances and use
       the first reachable instance for this purpose. (Bug
       #25267603)

     * The dba.configureInstance() operation was not setting the
       binlog_checksum option with the required value (NONE) in
       the option file for instances that did not support SET
       PERSIST (for example instances running MySQL 5.7), when
       the option file path was not provided as an input
       parameter but instead specified though the operation
       wizard in interactive mode. (Bug #96489, Bug #30171090)

Functionality Added or Changed


     * MySQL Shell’s upgrade checker utility (the
       util.checkForServerUpgrade() operation) includes the
       following new and extended checks:

          + The utility now checks for tablespace names
            containing the string “FTS”, which can be
            incorrectly identified as tablespaces of full-text
            index tables, preventing upgrade. The issue has been
            fixed in MySQL 8.0.18, but affects upgrades to
            earlier MySQL 8.0 releases.

          + The check for database objects with names that
            conflict with reserved keywords now covers the
            additional keywords ARRAY, MEMBER, and LATERAL.

          + – The checks for obsolete sql_mode flags now check
            the global sql_mode setting.
       Running the upgrade checker utility no longer alters the
       gtid_executed value, meaning that the utility can be used
       on Group Replication group members without affecting
       their synchronization with the group. The upgrade checker
       also now works correctly with the ANSI_QUOTES SQL mode.
       (Bug #30002732, Bug #30103683, Bug #96351)
       References: See also: Bug #29992589.

     * MySQL Shell has two new built-in reports, which provide
       information drawn from various sources including MySQL’s
       Performance Schema:

          + threads lists the current threads in the connected
            MySQL server which belong to the user account that
            is used to run the report. Using the report-specific
            options, you can choose to show foreground threads,
            background threads, or all threads. You can report a
            default set of information for each thread, or
            select specific information to include in the report
            from a larger number of available choices. You can
            filter, sort, and limit the output.

          + thread provides detailed information about a
            specific thread in the connected MySQL server. By
            default, the report shows information on the thread
            used by the current connection, or you can identify
            a thread by its ID or by the connection ID. You can
            select one or more categories of information, or
            view all of the available information about the
            thread.
       You can run the new reports using MySQL Shell’s \show and
       \watch commands. The reports work with servers running
       all supported MySQL 5.7 and MySQL 8.0 versions. If any
       item of information is not available in the MySQL Server
       version of the target server, the reports leave it out.

     * MySQL Shell has two new control commands:

          + The \edit (\e) command opens a command in the
            default system editor for editing. If you specify an
            argument to the command, this text is placed in the
            editor, and if you do not, the last command in the
            MySQL Shell history is placed in the editor. When
            you have finished editing, MySQL Shell presents your
            edited text ready for you to execute or cancel. The
            command can also be invoked using the short form \e
            or the key combination Ctrl-X Ctrl-E.

          + The \system (\!) command runs the operating system
            command that you specify as an argument to the
            command, then displays the output from the command
            in MySQL Shell. MySQL Shell returns an error if it
            was unable to execute the command.

     * MySQL Shell now uses Python 3. For platforms that include
       a system supported installation of Python 3, MySQL Shell
       uses the most recent version available, with a minimum
       supported version of Python 3.4.3. For platforms where
       Python 3 is not included, MySQL Shell bundles Python
       3.7.4. MySQL Shell maintains code compatibility with
       Python 2.6 and Python 2.7, so if you require one of these
       older versions, you can build MySQL Shell from source
       using the appropriate Python version.

Bugs Fixed


     * In debug mode, MySQL Shell raised an assertion when
       handling a character contained in SQL strings. (Bug
       #30286680)

     * If a Python lambda was added as a member of a MySQL Shell
       extension object, the Python object was not released
       correctly when MySQL Shell shut down, causing a
       segmentation fault. (Bug #30156304)

     * A memory leak could occur when Python code was executed
       in interactive mode. (Bug #30138755)

     * Help information for a MySQL Shell report could not be
       displayed unless there was an active session. MySQL Shell
       now checks for an open session only before actually
       running the report. (Bug #30083371)

     * If a default schema was set for the MySQL Shell
       connection, and a different default schema was set after
       the connection was made, MySQL Shell’s \reconnect command
       attempted to use the default schema from the original
       connection. The user’s current default schema is now used
       for the reconnection attempt. (Bug #30059354)

     * Due to a bug introduced by a change in MySQL Shell
       8.0.16, the MSI file that is used by Windows Installer to
       install MySQL Shell overwrote the Windows PATH
       environment variable with the path to the application
       binary (mysqlsh), removing any other paths present. The
       issue has now been fixed. (Bug #29972020, Bug #95432)

     * When the \reconnect command is used to attempt
       reconnection to a server, if the last active schema set
       by the user appears to be no longer available, MySQL
       Shell now attempts to connect with no schema set. (Bug
       #29954572)

     * In interactive mode, MySQL Shell now handles multiline
       comments beginning with a slash and asterisk (/*) and
       ending with an asterisk and slash (*/). (Bug #29938424)

     * The MySQL Shell \source command was not handled correctly
       when used in combination with SQL statements. (Bug
       #29889926)

     * With MySQL Shell in SQL mode, if multiple SQL statements
       including a USE statement were issued on a single line
       with delimiters, the USE statement was not handled
       correctly. (Bug #29881615)

     * If MySQL Shell’s JSON import utility was used to send a
       large number of JSON documents to a server with
       insufficient processing capacity, the utility could fill
       up the write queue with batches of prepared documents,
       causing the connection to time out and the import to
       fail. The utility now waits to read the response from the
       server before sending the next batch of prepared
       documents to the server. (Bug #29878964)

     * When MySQL Shell was built from source with a bundled
       OpenSSL package, the required linker flags were not set.
       The issue has now been fixed. (Bug #29862189)

     * If a new query was executed in MySQL Shell while a result
       was still active, resulting in rows being cached, not all
       rows were returned by the old query. (Bug #29818714)


On Behalf of Oracle/MySQL Release Engineering Team,
Balasubramanian Kandasamy

MySQL Shell 8.0.17 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.17 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.17.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster, providing an
integrated solution for high availability and scalability using InnoDB
based MySQL databases, without requiring advanced MySQL expertise. For
more information about how to configure and work with MySQL InnoDB
cluster see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.  For more
information about how to use MySQL Shell and the MySQL Document Store
support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.17, see the
“Generally Available (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!


Changes in MySQL Shell 8.0.17 (2019-07-22, General Availability)


InnoDB Cluster Added or Changed Functionality


     * Important Change: The handling of internal recovery
       accounts created by InnoDB cluster has been changed so that by
       default accounts are always created as
       “mysql_innodb_cluster_server_id@%”, where server_id is instance
       specific. This generated recovery account name is stored in the
       InnoDB cluster metadata, to ensure the correct account is always
       removed if the instance is removed from the cluster.  The
       previous behavior where multiple accounts would be created if
       ipWhitelist was given has been removed. In addition
       Cluster.removeInstance() no longer removes all recovery accounts
       on the instance being removed. It now removes the recovery
       account of the instance being removed on the primary and waits
       for the changes to be replicated before actually removing the
       instance from the group. Similarly, Cluster.rejoinInstance() no
       longer drops any recovery accounts. It only creates the recovery
       account of the instance being rejoined if it no longer exists on
       the primary (which it should in normal circumstances). If the
       recovery account already exists, it is reused by
       Cluster.rejoinInstance().  When a cluster is adopted from an
       existing Group Replication deployment, new recovery accounts are
       created and set for each member. Pre-existing accounts configured
       by the user are left unchanged and not dropped, unless they have
       the “mysql_innodb_cluster_” prefix.  As part of this work, the
       behavior of dba.createCluster() and
       Cluster.rebootClusterFromCompleteOutage() operations has been
       changed. Now, if these operations encounter an instance which has
       super_read_only=ON, it is disabled automatically. Therefore the
       clearReadOnly option has been deprecated for these operations.
       References: See also: Bug #29629121, Bug #29559303.

     * The dba.createCluster() operation has been improved, and
       as part of this work the order in which some steps of the
       operation are executed was changed. Now, the creation of the
       recovery (replication) user and updates to the Metadata are
       performed after bootstrapping the Group Replication group. As
       part of this work, the dba.createCluster() operation has been
       updated to support the interactive option, which is a boolean
       value that controls the wizards provided. When interactive is
       true, prompts and confirmations are displayed by the operation.
       The default value of interactive is equal to useWizards option.

     * The compatibility policies that Group Replication
       implements for member versions in groups now consider the patch
       version of a member’s MySQL Server release. Previously, when
       combining instances running different MySQL versions, only the
       major version was considered.  InnoDB cluster has been updated to
       support cluster operations where these compatibility policies
       have an impact. Using the patch version ensures better
       replication safety for mixed version groups during group
       reconfiguration and upgrade procedures. As part of this work the
       information provided about instances has been extended.  The
       following InnoDB cluster changes have been made to support the
       compatibility policies:

          + The Cluster.addInstance() operation now detects
            incompatibilities due to MySQL versions and in the
            event of an incompatibility aborts with an
            informative error.

          + The Cluster.status() attribute mode now considers
            the value of super_read_only and whether the cluster
            has quorum.

          + The Cluster.status() output now includes the boolean
            attribute autoRejoinRunning, which is displayed per
            instance belonging to the cluster and is true when
            automatic rejoin is running.

          + The extended option has been changed to accept
            integer or Boolean values. This makes the behavior
            similar to the queryMembers option, so that option
            has now been deprecated.
       References: See also: Bug #29557250.

     * InnoDB cluster supports the new MySQL Clone plugin on
       instances running 8.0.17 and later. When an InnoDB cluster is
       configured to use MySQL Clone, instances which join the cluster
       choose whether to use Group Replication’s distributed recovery or
       MySQL Clone to recover the transactions processed by the cluster.
       You can optionally configure this behavior, for example to force
       cloning, which replaces any transactions already processed. You
       can also configure how Cluster.addInstance() behaves, letting
       cloning operations proceed in the background or showing different
       levels of progress in MySQL Shell. This enables you to
       automatically provision instances in the most efficient way. In
       addition, the output of Cluster.status() for members in
       RECOVERING state has been extended to include recovery progress
       information to enable you to easily monitor recovery operations,
       whether they be using MySQL Clone or distributed recovery.

InnoDB Cluster Bugs Fixed


     * Important Change: The sandboxes deployed using the
       AdminAPI did not support the RESTART statement. Now, the wrapper
       scripts call mysqld in a loop so that there is a monitoring
       process which ensures that RESTART is supported. (Bug #29725222)

     * The Cluster.addInstance() operation did not validate if
       the server_id of the joining instance was not unique among all
       cluster members. Although the use of a unique server_id is not
       mandatory for Group Replication to work properly (because all
       internal replication channels use –replicate-same-server-id=ON),
       it was recommended that all instances in a replication stream
       have a unique server_id. Now, this recommendation is a
       requirement for InnoDB cluster, and when you use the
       Cluster.addInstance() operation if the server_id is already used
       by an instance in the cluster then the operation fails with an
       error. (Bug #29809560)

     * InnoDB clusters do not support instances that have binary
       log filters configured, but replication filters were being
       allowed. Now, instances with replication filters are also blocked
       from InnoDB cluster usage. (Bug #29756457) References: See also:
       Bug #28064729, Bug #29361352.

     * On instances running version 8.0.16, the
       Cluster.rejoinInstance() operation failed when one or more
       cluster members were in RECOVERING state, because the Group
       Replication communication protocol could not be obtained. More
       specifically, the group_replication_get_communication_protocol()
       User-Defined function (UDF) failed because it could only be
       executed if all members were ONLINE. Now, in the event of the UDF
       failing when rejoining an instance a warning is displayed and
       AdminAPI proceeds with the execution of the operation.  Starting
       from MySQL 8.0.17, the
       group_replication_get_communication_protocol() UDF no longer
       issues an error if a member is RECOVERING. (Bug #29754915)

     * On Debian-based hosts, hostname resolves to the IP
       address 127.0.1.1 by default, which does not match a real network
       interface. This is not supported by Group Replication, which made
       sandboxes deployed on such hosts unusable unless a manual change
       to the configuration file was made. Now, the sandbox
       configuration files created by MySQL Shell contain the following
       additional line: report_host = 127.0.0.1 In other words the
       report_host variable is set to the loopback IP address. This
       ensures that sandbox instances can be used on Debian-based hosts
       without any additional manual changes. (Bug #29634828)

     * If the binary logs had been purged from all cluster
       instances, Cluster.checkInstanceState() lacked the ability to
       check the instance’s state, resulting in erroneous output values.
       Now, Cluster.checkInstanceState() validates the value of
       GTID_PURGED on all cluster instances and provides the correct
       output and also an informative message mentioning the possible
       actions to be taken. In addition, Cluster.addInstance() and
       Cluster.rejoinInstance() were not using the checks performed by
       Cluster.checkInstanceState() in order to verify the GTID status
       of the target instance in relation to the cluster.  In the event
       of all cluster instances having their binary logs purged, the
       Cluster.addInstance() command would succeed but the instance
       would never be able to join the cluster as distributed recovery
       failed to execute. Now, both operations make use of the checks
       performed by Cluster.checkInstanceState() and provide informative
       error messages. (Bug #29630591, Bug #29790569)

     * When using the dba.configureLocalInstance() operation in
       interactive mode, if you provided the path to an option file it
       was ignored. (Bug #29554251)

     * Calling cluster.removeInstance() on an instance that did
       not exist, for example due to a typo or because it was already
       removed, resulted in a prompt asking whether the instance should
       be removed anyway, and the operation then failing.
       (Bug #29540529)

     * To use an instance for InnoDB cluster, whether it is to
       create a cluster on it or add it to an existing cluster, requires
       that the instance is not already serving as a slave in
       asynchronous (master-slave) replication. Previously,
       dba.checkInstanceConfiguration() incorrectly reported that a
       target instance which was running as an asynchronous replication
       slave as valid for InnoDB cluster usage. As a consequence,
       attempting to use such instances with operations such as
       dba.createCluster() and Cluster.addInstance() failed without
       informative errors.  Now, dba.checkInstanceConfiguration()
       verifies if the target instance is already configured as a slave
       using asynchronous replication and generates an error if that is
       the case. Similarly, the dba.createCluster(),
       Cluster.addInstance(), and Cluster.rejoinInstance() operations
       detect such instances and block them from InnoDB cluster usage.
       Note that this does not prevent instances which belong to a
       cluster also functioning as the master in asynchronous
       replication. (Bug #29305551)

     * The dba.createCluster() operation was allowed on a target
       instance that already had a populated Metadata schema, when the
       instance was already in that Metadata. The Metadata present on
       the target instance was being overridden, which was unexpected.
       Now, in such a situation the dba.createCluster() throws an
       exception and you can choose to either drop the Metadata schema
       or reboot the cluster. (Bug #29271400)

     * When a sandbox instance of MySQL had been successfully
       started from MySQL Shell using dba.startSandboxInstance(),
       pressing Ctrl+C in the same console window terminated the sandbox
       instance. Sandbox instances are now launched in a new process
       group so that they are not affected by the interrupt.
       (Bug #29270460)

     * During the creation of a cluster using the AdminAPI, some
       internal replication users are created with user names which
       start with “mysql_innodb_cluster”. However, if the MySQL server
       had a global password expiration policy defined, for example if
       default_password_lifetime was set to a value other than zero,
       then the passwords for the internal users expired after reaching
       the specified period. Now, the internal user accounts are created
       by the AdminAPI with password expiration disabled.
       (Bug #28855764)

     * The dba.checkInstanceConfiguration() and
       dba.configureInstance() operations were not checking the validity
       of persisted configurations, which can be different from the
       corresponding system variable value, in particular when changed
       with SET PERSIST_ONLY. This could lead these operations to report
       wrong or inaccurate results, for example reporting that the
       instance configuration is correct when in reality the persisted
       configuration was invalid and wrong settings could be applied
       after a restart of the server, or inaccurately reporting that a
       server update was needed when only a restart was required. (Bug
       #28727505) References: See also: Bug #29765093.

     * When you removed an instance’s metadata from a cluster
       without removing the metadata from the instance itself (for
       example because of wrong authentication or when the instance was
       unreachable) the instance could not be added again to the
       cluster. Now, another validation has been added to
       Cluster.addInstance() to verify if the instance already belongs
       to the cluster’s underlying group but is not in the InnoDB
       cluster metadata, issuing an error if it already belongs to the
       ReplicaSet. Similarly, an error is issued when the default port
       automatically set for the local address is invalid (out of range)
       instead of using a random port. (Bug #28056944)

     * When issuing dba.configureInstance() in interactive mode
       and after selecting option number 2 “Create a new admin account
       for InnoDB cluster with minimal required grants” it was not
       possible to enter a password for the new user.

Functionality Added or Changed


     * MySQL Shell has a new function for SQL query execution
       for X Protocol sessions that works in the same way as the
       function for SQL query execution in classic MySQL protocol
       sessions. The new function, Session.runSql(), can be used in
       MySQL Shell only as an alternative to X Protocol’s Session.sql()
       to create a script that is independent of the protocol used for
       connecting to the MySQL server. Note that Session.runSql() is
       exclusive to MySQL Shell and is not part of the standard X
       DevAPI. As part of this change, the ClassicSession.query function
       for SQL query execution, which is a synonym of
       ClassicSession.runSQL(), is now deprecated.  A new function
       fetchOneObject() is also provided for the classic MySQL protocol
       and X Protocol to return the next result as a scripting object.
       Column names are used as keys in the dictionary (and as object
       attributes if they are valid identifiers), and row values are
       used as attribute values in the dictionary. This function enables
       the query results to be browsed and used in protocol-independent
       scripts. Updates made to the returned object are not persisted on
       the database.

     * MySQL Shell’s new parallel table import utility provides
       rapid data import to a MySQL relational table for large data
       files. The utility analyzes an input data file, divides it into
       chunks, and uploads the chunks to the target MySQL server using
       parallel connections. The utility is capable of completing a
       large data import many times faster than a standard
       single-threaded upload using a LOAD DATA statement.  When you
       invoke the parallel table import utility, you specify the mapping
       between the fields in the data file and the columns in the MySQL
       table. You can set field- and line-handling options as for the
       LOAD DATA command to handle data files in arbitrary formats. The
       default dialect for the utility maps to a file created using a
       SELECT … INTO OUTFILE statement with the default settings for
       that statement. The utility also has preset dialects that map to
       the standard data formats for CSV files (created on DOS or UNIX
       systems), TSV files, and JSON, and you can customize these using
       the field- and line-handling options as necessary.

     * MySQL Shell has a number of new display options for query
       results:

          + The shell.dumpRows() function can format a result
            set returned by a query in any of the output formats
            supported by MySQL Shell, and dump it to the
            console. Note that the result set is consumed by the
            function. This function can be used in MySQL Shell
            to display the results of queries run by scripts to
            the user in the same ways as the interactive SQL
            mode can.

          + The new MySQL Shell output format json/array
            produces raw JSON output wrapped in a JSON array.
            The output format ndjson is added as a synonym for
            json/raw, and both those output formats produce raw
            JSON output delimited by newlines. You can select
            MySQL Shell output formats by starting MySQL Shell
            with the –result-format=[value] command line
            option, or setting the MySQL Shell configuration
            option resultFormat.
       A new function shell.unparseUri() is also added, which converts a
       dictionary of URI components and connection options into a valid
       URI string for connecting to MySQL.

     * You can now extend MySQL Shell with plugins that are
       loaded at startup. MySQL Shell plugins can be written in either
       JavaScript or Python, and the functions they contain are
       available in MySQL Shell in both JavaScript and Python modes. The
       plugins can be used to contain functions that are registered as
       MySQL Shell reports, and functions that are members of extension
       objects that are made available as part of user-defined MySQL
       Shell global objects.  You can create a MySQL Shell plugin by
       storing code in a subfolder of the plugins folder in the MySQL
       Shell user configuration path, with an initialization file that
       MySQL Shell locates and executes at startup. You can structure a
       plugin group, with a collection of related plugins that can share
       common code, by placing the subfolders for multiple plugins in a
       containing folder under the plugins folder.

     * You can now extend the base functionality of MySQL Shell
       by defining extension objects and making them available as part
       of additional MySQL Shell global objects. Extension objects can
       be written in JavaScript or Python.  When you create and register
       an extension object, it is available in MySQL Shell in both
       JavaScript and Python modes. You construct and register extension
       objects using functions provided by the built-in global object
       shell.

     * You can now configure MySQL Shell to send logging
       information to the console, in addition to sending it to the
       application log. The –verbose command-line option and the
       verbose MySQL Shell configuration option activate this function.
       By default, when the option is set, internal error, error,
       warning, and informational messages are sent to the console,
       which is the equivalent to a logging level of 5 for the
       application log. You can add three further levels of debug
       messages, up to the highest level of detail.

     * MySQL Shell’s upgrade checker utility (the
       util.checkForServerUpgrade() operation) carries out two new
       checks. When checking for upgrade from any MySQL 5.7 release to
       any MySQL 8.0 release, the utility identifies partitioned tables
       that use storage engines other than InnoDB or NDB and therefore
       rely on generic partitioning support from the MySQL server, which
       is no longer provided. When checking for upgrade from any release
       to MySQL 8.0.17, the utility identifies circular directory
       references in tablespace data file paths, which are no longer
       permitted.

     * X DevAPI now supports indexing array fields. A single
       index field description can contain a new member name array that
       takes a Boolean value. If set to true, the field is assumed to
       contain arrays of elements of the given type. In addition, the
       set of possible index field data types (used as values of member
       type in index field descriptions) is extended with type CHAR(N),
       where the length N is mandatory.

     * MySQL Shell now supports the ability to send connection
       attributes (key-value pairs that application programs can pass to
       the server at connect time). MySQL Shell defines a default set of
       attributes, which can be disabled or enabled. In addition,
       applications can specify attributes to be passed in addition to
       the default attributes. The default behavior is to send the
       default attribute set.  You specify connection attributes as a
       connection-attributes parameter in a connection string.  The
       connection-attributes parameter value must be empty (the same as
       specifying true), a Boolean value (true or false to enable or
       disable the default attribute set), or a list or zero or more
       key=value specifiers separated by commas (to be sent in addition
       to the default attribute set). Within a list, a missing key value
       evaluates as an empty string. Examples:
       “mysqlx://user@host?connection-attributes”
       “mysqlx://user@host?connection-attributes=true”
“mysqlx://user@host?connection-attributes=false”
“mysqlx://user@host?connection-attributes=[attr1=val1,attr2,attr3=]”
       “mysqlx://user@host?connection-attributes=[]”

       You can specify connection attributes for both X Protocol
       connections and MySQL classic protocol connections. The
       default attributes set by MySQL Shell are:
> \sql SELECT ATTR_NAME, ATTR_VALUE FROM performance_schema.session_ac
count_connect_attrs;
+———————–+——————–+
| ATTR_NAME       | ATTR_VALUE |
+———————–+——————–+
| _pid            | 28451      |
| _platform       | x86_64     |
| _os             | Linux      |
| _client_name    | libmysql   |
| _client_version | 8.0.17     |
| program_name    | mysqlsh    |
+———————-+——————–+

       Application-defined attribute names cannot begin with _ because
       such names are reserved for internal attributes.  If connection
       attributes are not specified in a valid way, an error occurs and
       the connection attempt fails.  For general information about
       connection attributes, see Performance Schema Connection
       Attribute Tables
 (https://dev.mysql.com/doc/refman/8.0/en/performance-schema-connection-attribute-tables.html).

     * MySQL Shell now supports the OVERLAPS and NOT OVERLAPS
       operators for expressions on JSON arrays or objects:
expr OVERLAPS expr
expr NOT OVERLAPS expr

       These operators behave in a similar way to the
       JSON_OVERLAPS() function. Suppose that a collection has
       these contents:
mysql-js> myCollection.add([{ “_id”: “1”, “list”: [1, 4] }, { “_id”: ”
2″, “list”: [4, 7] }])

       This operation:
mysql-js> var res = myCollection.find(“[1, 2, 3] OVERLAPS $.list”).fie
lds(“_id”).execute();
mysql-js> res

       Should return:
{
“_id”: “1”
}
1 document in set (0.0046 sec)

       This operation:
mysql-js> var res = myCollection.find(“$.list OVERLAPS [4]”).fields(“_
id”).execute();
mysql-js> res

       Should return:
{
“_id”: “1”
}
{
“_id”: “2”
}
2 documents in set (0.0031 sec)

       An error occurs if an application uses either operator
       and the server does not support it.

Bugs Fixed


     * With MySQL Shell in Python mode, using auto-completion on
       a native MySQL Shell object caused informational messages about
       unknown attributes to be written to the application log file.
       (Bug #29907200)

     * The execution time for statements issued in MySQL Shell
       in multiple-line mode has been reduced by reparsing the code only
       after the delimiter is found. (Bug #29864587)

     * Python’s sys.argv array was only initialized when MySQL
       Shell was started in batch mode, and was not initialized when
       MySQL Shell was started in interactive mode. (Bug #29811021)

     * MySQL Shell incorrectly encoded the CAST operation as a
       function call rather than a binary operator, resulting in SQL
       syntax errors. (Bug #29807711)

     * MySQL Shell now supports the unquoting extraction
       operator ->> for JSON. (Bug #29794340)

     * Handling of empty lines in scripts processed by MySQL
       Shell in batch mode has been improved. (Bug #29771369)

     * On Windows, when a MySQL Shell report was displayed using
       the \watch command, pressing Ctrl+C to interrupt execution of the
       command did not take effect until the end of the refresh interval
       specified with the command.  The interrupt now takes effect
       immediately. Also, any queries executed by reports run using the
       \show or \watch commands are now automatically cancelled when
       Ctrl+C is pressed. (Bug #29707077)

     * In Python mode, native dictionary objects created by
       MySQL Shell did not validate whether they contained a requested
       key, which could result in random values being returned or in a
       SystemError exception being thrown. Key validation has now been
       added, and a KeyError exception is thrown if an invalid key is
       requested. (Bug #29702627)

     * When using MySQL Shell in interactive mode, if raw JSON
       output was being displayed from a source other than a terminal
       (for example a file or a pipe), in some circumstances the prompt
       was shown on the same line as the last line of the output. The
       issue has now been corrected, and a new line is printed before
       the prompt message if the last line of the output did not end
       with one. (Bug #29699640)

     * The MySQL Shell \sql command, which executes a single SQL
       statement while another language is active, now supports the \G
       statement delimiter to print result sets vertically.
       (Bug #29693853)

     * Some inconsistencies in MySQL Shell’s choice of stdout or
       stderr for output have been corrected, so that only expected
       output that is intended to be processed by other programs goes to
       stdout, and all informational messages, warnings, and errors go
       to stderr. (Bug #29688637)

     * When MySQL Shell was started with the option
       –quiet-start=2 to print only error messages, warning messages
       about the operation of the upgrade checker utility
       checkForServerUpgrade() were still printed. (Bug #29620947)

     * In Python mode, native dictionary objects created by
       MySQL Shell did not provide an iterator, so it was not possible
       to iterate over them or use them with the in keyword.
       Functionality to provide Python’s iterator has now been added.
       (Bug #29599261)

     * When a MySQL Shell report was displayed using the \watch
       command, the screen was cleared before the report was rerun. With
       a report that executed a slow query, this resulted in a blank
       screen being displayed for noticeable periods of time. The screen
       is now cleared just before the report generates its first text
       output. (Bug #29593246)

     * MySQL Shell’s upgrade checker utility
       checkForServerUpgrade() returned incorrect error text for each
       removed system variable that was detected in the configuration
       file. (Bug #29508599)

     * MySQL Shell would hang when attempting to handle output
       from a stored procedure that produced results repeatedly from a
       single statement. The issues have now been corrected. (Bug
       #29451154, Bug #94577)

     * You can now specify the command line option –json to
       activate JSON wrapping when you start MySQL Shell to use the
       upgrade checker utility. In this case, JSON output is returned as
       the default, and you can choose raw JSON format by specifying
       –json=raw. Also, warning and error messages relating to running
       the utility have been removed from the JSON output.
       (Bug #29416162)

     * In SQL mode, when MySQL Shell was configured to use an
       external pager tool to display output, the pager was invoked
       whether or not the query result was valid. For an invalid query,
       this resulted in the pager displaying an empty page, and the
       error message was only visible after quitting the pager. The
       pager tool is now only invoked when a query returns a valid
       result, otherwise the error message is displayed.
       (Bug #29408598, Bug #94393)

     * MySQL Shell did not take the ANSI_QUOTES SQL mode into
       account when parsing quote characters. (Bug #27959072)

     * Prompt theme files for MySQL Shell that were created on
       Windows could not be used on other platforms. The issue, which
       was caused by the parser handling the carriage return character
       incorrectly, has now been fixed. (Bug #26597468)

     * The use of the mysqlsh command-line option –execute (-e)
       followed by –file (-f) when starting MySQL Shell is now
       disallowed, as these options are mutually exclusive. If the
       options are specified in that order, an error is returned. Note
       that if –file is specified first, –execute is treated as an
       argument of the processed file, so no error is returned.
       (Bug #25686324)

     * Syntax errors returned by MySQL Shell’s JavaScript
       expression parser have been improved to provide context and
       clarify the position of the error. (Bug #24916806)

On Behalf of Oracle/MySQL Release Engineering Team,
Nawaz Nazeer Ahamed

MySQL Shell 8.0.16 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.16 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.16.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster, providing an
integrated solution for high availability and scalability using InnoDB
based MySQL databases, without requiring advanced MySQL expertise. For
more information about how to configure and work with MySQL InnoDB
cluster see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language. For more
information about how to use MySQL Shell and the MySQL Document Store
support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.16, see the
“Generally Available (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

 

On Behalf of Oracle/MySQL Release Engineering Team,
Nawaz Nazeer Ahamed

MySQL Shell 8.0.15 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.15 is a maintenance release of MySQL Shell 8.0 Series
(a component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0
and 5.7. Please upgrade to MySQL Shell 8.0.15.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a
document store.

The AdminAPI enables you to work with MySQL InnoDB cluster, providing
an integrated solution for high availability and scalability using
InnoDB based MySQL databases, without requiring advanced MySQL
expertise. For more information about how to configure and work with
MySQL InnoDB cluster see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.
For more information about how to use MySQL Shell and the MySQL Document
Store support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html.

For more information on the APIs provided with MySQL Shell
see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.15, see
the “Generally Available (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

On Behalf of Oracle/MySQL Release Engineering Team,
Balasubramanian Kandasamy

MySQL Shell 8.0.14 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.14 is a maintenance release of MySQL Shell 8.0 Series
(a component of the MySQL Server). The MySQL Shell is provided under Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and 5.7. Please upgrade to MySQL Shell 8.0.14.

MySQL Shell is an interactive JavaScript, Python and SQL console interface, supporting development and administration for the MySQL Server. It provides APIs implemented in JavaScript and Python that enable you to work with MySQL InnoDB cluster and use MySQL as a document store.

The AdminAPI enables you to work with MySQL InnoDB cluster, providing an integrated solution for high availability and scalability using InnoDB based MySQL databases, without requiring advanced MySQL expertise. For more information about how to configure and work with MySQL InnoDB cluster see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document collections and perform Create, Update, Read, Delete (CRUD) operations on those collections from your favorite scripting language.
For more information about how to use MySQL Shell and the MySQL Document Store support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI you can also use the latest MySQL Connectors for your language of choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the legacy MySQL protocol. Additionally, MySQL Shell provides partial compatibility with the mysql client by supporting many of the same command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.14, see the “Generally Available (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

==================================================

On Behalf of Oracle/MySQL Release Engineering Team,
Kent Boortz