MySQL Shell 8.0.21 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.21 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.21.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster and InnoDB
ReplicaSet, providing integrated solutions for high availability and scalability
using InnoDB based MySQL databases, without requiring advanced MySQL
expertise.  For more information about how to configure and work with
MySQL InnoDB cluster and MySQL InnoDB ReplicaSet see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.  For more
information about how to use MySQL Shell and the MySQL Document Store
support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.21, see the
“General Availability (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

Changes in MySQL Shell 8.0.21 (2020-07-13, General Availability)

     * AdminAPI Added or Changed Functionality
     * AdminAPI Bugs Fixed
     * Functionality Added or Changed
     * Bugs Fixed

AdminAPI Added or Changed Functionality

     * A new user configurable tag framework has been added to
       the metadata, to allow specific instances of a cluster or
       ReplicaSet to be marked with additional information. Tags
       can be any ASCII character and provide a namespace. You
       set tags for an instance using the setInstanceOption()
       operation. In addition, AdminAPI and MySQL Router 8.0.21
       support specific tags, which enable you to mark instances
       as hidden and remove them from routing. MySQL Router then
       excludes such tagged instances from the routing
       destination candidates list. This enables you to safely
       take a server instance offline, so that applications and
       MySQL Router ignore it, for example while you perform
       maintenance tasks, such as server upgrade or
       configuration changes. To bring the instance back online,
       use the setInstanceOption() operation to remove the tags
       and MySQL Router adds the instance back to the routing
       destination candidates list, and it becomes online for
       applications. For more information, see Tagging the
       Metadata
(https://dev.mysql.com/doc/refman/8.0/en/admin-api-tagging.html).

AdminAPI Bugs Fixed

     * Important Change: Previously, Group Replication did not
       support binary log checksums, and therefore one of the
       requirements for instances in InnoDB cluster was that
       binary log checksums were disabled by having the
       binlog_checksum system variable set to NONE. AdminAPI
       verified the value of binlog_checksum during the
       dba.checkInstanceConfiguration() operation and disallowed
       creating a cluster or adding an instance to a cluster
       that did not have binary log checksums disabled. In
       version 8.0.21, Group Replication has lifted this
       restriction, therefore InnoDB cluster now permits
       instances to use binary log checksums, with
       binlog_checksum set to CRC32. The setting for
       binlog_checksum does not have to be the same for all
       instances. In addition, sandboxes deployed with version
       8.0.21 and later do not set the binlog_checksum variable,
       which defaults to CRC32. (Bug #31329024)

     * Adopting a Group Replication setup as a cluster can be
       performed when connected to any member of the group,
       regardless of whether it is a primary or a secondary.
       However, when a secondary member was used,
       super_read_only was being incorrectly disabled on that
       instance. Now, all operations performed during an
       adoption are done using the primary member of the group.
       This ensures that no GTID inconsistencies occur and that
       super_read_only is not incorrectly disabled on secondary
       members. (Bug #31238233)

     * Using the clusterAdmin option to create a user which had
       a netmask as part of the host resulted in an error when
       this user was passed to the dba.createCluster()
       operation. Now, accounts that specify a netmask are
       treated as accounts with wildcards, meaning that further
       checks to verify if the account accepts remote
       connections from all instances are skipped. (Bug
       #31018091)

     * The check for instance read-only compatibility was using
       a wrong MySQL version as the base version. The
       cross-version policies were added to Group Replication in
       version 8.0.17, but the check was considering instances
       running 8.0.16. This resulted in a misleading warning
       message indicating that the added instance was read-only
       compatible with the cluster, when this was not true (only
       for instances 8.0.16). The fix ensures that the check to
       verify if an instance is read-compatible or not with a
       cluster is only performed if the target instance is
       running version 8.0.17 or later. (Bug #30896344)

     * The maximum number of instances in an InnoDB cluster is
       9, but AdminAPI was not preventing you from trying to add
       more instances to a cluster and the resulting error
       message was not clear. Now, if a cluster has 9 instances,
       Cluster.addInstance prevents you adding more instances.
       (Bug #30885157)

     * Adding an instance with a compatible GTID set to a InnoDB
       cluster or InnoDB ReplicaSet on which provisioning is
       required should not require any interaction, because this
       is considered a safe operation. Previously, in such a
       scenario, when MySQL Clone was supported MySQL Shell
       still prompted to choose between cloning or aborting the
       operation. Now, the operation proceeds with cloning,
       because this is the only way to provision the instance.
       Note
       instances with an empty GTID set are not considered to
       have a compatible GTID set when compared with the InnoDB
       cluster or InnoDB ReplicaSet. Such scenarios are
       considered to be unknown, therefore MySQL Shell prompts
       to confirm which action should be taken.
       (Bug #30884590)

     * The Group Replication system variables (prefixed with
       group_replication) do not exist if the plugin has not
       been loaded. Even if the system variables are persisted
       to the instance’s option file, they are not loaded unless
       the Group Replication plugin is also loaded when the
       server starts. If the Group Replication plugin is
       installed after the server starts, the option file is not
       reloaded, so all system variables have default values.
       Instances running MySQL 8.0 do not have a problem because
       SET PERSIST is used. However, on instances running
       version MySQL 5.7, the dba.rebootCluster() operation
       could not restore some system variables if the Group
       Replication plugin was uninstalled. Now, the
       dba.configureInstance() operation persists the Group
       Replication system variables to configuration files with
       the loose_ prefix. As a result, once the Group
       Replication plugin is installed, on instances running 5.7
       the persisted values are used instead of the default
       values. (Bug #30768504)

     * The updateTopologyMode option has been deprecated and the
       behavior of Cluster.rescan() has been changed to always
       update the topology mode in the Metadata when a change is
       detected. MySQL Shell now displays a message whenever
       such a change is detected. (Bug #29330769)

     * The cluster.addInstance() and cluster.rejoinInstance()
       operations were not checking for the full range of
       settings which are required for an instance to be valid
       for adding to the cluster. This resulted in attempts to
       use instances which run on different operating systems to
       fail. For example, a cluster running on two instances
       that were hosted on a Linux based operating system would
       block the addition of an instance running Microsoft
       Windows. Now, cluster.addInstance() and
       cluster.rejoinInstance() operations validate the instance
       and prevent adding or rejoining an instance to the
       cluster if the value of the lower_case_table_names,
       group_replication_gtid_assignment_block_size or
       default_table_encryption of the instance are different
       from the ones on the cluster. (Bug #29255212)

Functionality Added or Changed

     * MySQL Shell now has an instance dump utility,
       dumpInstance(), and schema dump utility, dumpSchemas().
       The new utilities support the export of all schemas or a
       selected schema from an on-premise MySQL server instance
       into an Oracle Cloud Infrastructure Object Storage bucket
       or a set of local files. The schemas can then be imported
       into a MySQL Database Service DB System using MySQL
       Shell’s new dump loading utility. The new utilities
       provide Oracle Cloud Infrastructure Object Storage
       streaming, MySQL Database Service compatibility checks
       and modifications, parallel dumping with multiple
       threads, and file compression.

     * MySQL Shell’s new dump loading utility, loadDump(),
       supports the import of schemas dumped using MySQL Shell’s
       new instance dump utility and schema dump utility into a
       MySQL Database Service DB System. The dump loading
       utility provides data streaming from remote storage,
       parallel loading of tables or table chunks, progress
       state tracking, resume and reset capability, and the
       option of concurrent loading while the dump is taking
       place.

     * The X DevAPI implementation now supports JSON schema
       validation, which enables you to ensure that your
       documents have a certain structure before they can be
       inserted or updated in a collection. To enable or modify
       JSON schema validation you pass in a JSON object like:
{
    validation: {
      level: “off|strict”,
      schema: “json-schema”
    }
}

       Here, validation is JSON object which contains the keys
       you can use to configure JSON schema validation. The
       first key is level, which can take the value strict or
       off. The second key, schema, is a JSON schema, as defined
       at http://json-schema.org. If the level key is set to
       strict, documents are validated against the json-schema
       when they are added to the collection, or when an
       operation updates the document. If the document does not
       validate, the server generates an error and the operation
       fails. If the level key is set to off, documents are not
       validated against the json-schema.
       You can pass a validation JSON object to the
       schema.createCollection() operation, to enable JSON
       schema validation, and schema.modifyCollection()
       operation, to change the current JSON schema validation,
       for example to disable validation. For more information,
       see JSON Schema Validation
(https://dev.mysql.com/doc/x-devapi-userguide/en/collection-validation.html).

Bugs Fixed

     * MySQL Shell plugins now support the use of the **kwargs
       syntax in functions defined in Python that are made
       available by the plugin. Using **kwargs in a function
       definition lets you call the function using a
       variable-length list of keyword arguments with arbitrary
       names. If the function is called from MySQL Shell’s
       JavaScript mode, MySQL Shell passes the named arguments
       and their values into a dictionary object for the Python
       function. MySQL Shell first tries to associate a keyword
       argument passed to a function with any corresponding
       keyword parameter that the function defines, and if there
       is none, the keyword argument is automatically included
       in the **kwargs list. As a side effect of this support,
       any API function called from Python in MySQL Shell that
       has a dictionary of options as the last parameter
       supports defining these options using named arguments.
       (Bug #31495448)

     * When switching to SQL mode, MySQL Shell queries the SQL
       mode of the connected server to establish whether the
       ANSI_QUOTES mode is enabled. Previously, MySQL Shell
       could not proceed if it did not receive a result set in
       response to the query. The absence of a result is now
       handled appropriately. (Bug #31418783, Bug #99728)

     * In SQL mode, when the results of a query are to be
       printed in table format, MySQL Shell buffers the result
       set before printing, in order to identify the correct
       column widths for the table. With very large result sets,
       it was possible for this practice to cause an out of
       memory error. MySQL Shell now buffers a maximum of 1000
       rows for a result set before proceeding to format and
       print the table. Note that if a field in a row after the
       first 1000 rows contains a longer value than previously
       seen in that column in the result set, the table
       formatting will be misaligned for that row. (Bug
       #31304711)

     * Context switching in MySQL Shell’s SQL mode has been
       refactored and simplified to remove SQL syntax errors
       that could be returned when running script files using
       the source command. (Bug #31175790, Bug #31197312, Bug
       #99303)

     * The user account that is used to run MySQL Shell’s
       upgrade checker utility checkForServerUpgrade()
       previously required ALL privileges. The user account now
       requires only the RELOAD, PROCESS, and SELECT privileges.
       (Bug #31085098)

     * In Python mode, MySQL Shell did not handle invalid UTF-8
       sequences in strings returned by queries. (Bug #31083617)

     * MySQL Shell’s parallel table import utility importTable()
       has a new option characterSet, which specifies a
       character set encoding with which the input data file is
       interpreted during the import. Setting the option to
       binary means that no conversion is done during the
       import. When you omit this option, the import uses the
       character set specified by the character_set_database
       system variable to interpret the input data file. (Bug
       #31057707)

     * On Windows, if the MySQL Shell package was extracted to
       and used from a directory whose name contained multi-byte
       characters, MySQL Shell was unable to start. MySQL Shell
       now handles directory names with multi-byte characters
       correctly, including when setting up Python, loading
       prompt themes, and accessing credential helpers. (Bug
       #31056783)

     * MySQL Shell’s JSON import utility importJSON() now
       handles UTF-8 encoded files that include a BOM (byte mark
       order) at the start, which is the sequence 0xEF 0xBB
       0xBF. As a workaround in earlier releases, remove this
       byte sequence, which is not needed. (Bug #30993547, Bug
       #98836)

     * When the output format was set to JSON, MySQL Shell’s
       upgrade checker utility checkForServerUpgrade() included
       a description and documentation link for a check even if
       no issues were found. These are now omitted from the
       output, as they are with the text output format. (Bug
       #30950035)

On Behalf of Oracle/MySQL Release Engineering Team,
Sreedhar S