MySQL Connector/J 8.0.17 has been released

Dear MySQL users,

MySQL Connector/J 8.0.17 is the latest General Availability
release of the MySQL Connector/J 8.0 series.  It is suitable
for use with MySQL Server versions 8.0, 5.7, and 5.6.
It supports the Java Database Connectivity (JDBC) 4.2 API,
and implements the X DevAPI.

This release includes the following new features and changes, also
described in more detail on

https://dev.mysql.com/doc/relnotes/connector-j/8.0/en/news-8-0-17.html

As always, we recommend that you check the “CHANGES” file in the
download archive to be aware of changes in behavior that might affect
your application.

To download MySQL Connector/J 8.0.17 GA, see the “Generally Available
(GA) Releases” tab at http://dev.mysql.com/downloads/connector/j/

Enjoy!

———————————————————————–

Changes in MySQL Connector/J 8.0.17 (2019-07-22, General
Availability)

Functionality Added or Changed

     * X DevAPI: The following methods have been deprecated:

          + Collection.find().where()

          + Collection.modify().where()

          + Collection.remove().where()

     * X DevAPI: Two new operators for JSON objects and arrays,
       overlaps and not_overlaps, are now supported. See the X
       DevAPI User Guide
       (https://dev.mysql.com/doc/x-devapi-userguide/en/) for
       details.

     * X DevAPI: Indexing for array fields is now supported. See
       Indexing Array Fields
       (https://dev.mysql.com/doc/x-devapi-userguide/en/collecti
       on-indexing.html#collection-indexing-array) in the X
       DevAPI User Guide
       (https://dev.mysql.com/doc/x-devapi-userguide/en/) for
       details.

     * The README and LICENSE files are now included inside the
       Connector/J JAR archive delivered in the
       platform-independent tarballs and zip files. (Bug
       #29591275)

     * A number of private parameters of ProfilerEvents (for
       example, hostname) had no getters for accessing them from
       outside of the class instance. Getter methods have now
       been added for all the parameters of the class. (Bug
       #20010454, Bug #74690)

     * A new connection property, databaseTerm, sets which of
       the two terms is used in an application to refer to a
       database. The property takes one of the two values
       CATALOG or SCHEMA and uses it to determine which
       Connection methods can be used to set/get the current
       database, which arguments can be used within the various
       DatabaseMetaData methods to filter results, and which
       fields in the ResultSet returned by DatabaseMetaData
       methods contain the database identification information.
       See the entry for databaseTerm in Configuration
       Properties
(https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-configuration-properties.html)
       for details. Also, the connection property
       nullCatalogMeansCurrent has been renamed to
       nullDatabaseMeansCurrent. The old name remains an alias
       for the connection property. Thanks to Harald Aamot for
       contributing to the patch.
       (Bug #11891000, Bug #27356869, Bug #89133)

     * A new CONTRIBUTING file has been added to the Connector/J
       repository on GitHub
       (https://github.com/mysql/mysql-connector-j), which
       provides guidelines for code contribution and bug
       reporting.

     * The MySQL Connector/J X DevAPI Reference can now be
       generated from the Connector/J source code as an Ant
       target, xdevapi-docs.

     * Added support for host names that are longer than 60
       characters (up to 255 characters), as they are now
       supported by MySQL Server 8.0.17.

     * Added support for the utf8mb4_0900_bin collation, which
       is now supported by MySQL Server 8.0.17.

     * A cached server-side prepared statement can no longer be
       effectively closed by calling Statement.close() twice. To
       close and de-cache the statement, do one of the
       following:

          + Close the connection (assuming the connection is
            tracking all open resources).

          + Use the implementation-specific method
            JdbcPreparedStatement.realClose().

          + Set the statement as non-poolable by calling the
            method Statement.setPoolable(false) before or after
            closing it.

Bugs Fixed

     * X DevAPI: The IN operator in X DevAPI expressions, when
       followed by a square bracket ([), got mapped onto the
       wrong operation in X Protocol. (Bug #29821029)

     * When using a replication connection, retrieving data from
       BlobFromLocator resulted in a ClassCastException. It was
       due to some wrong and unnecessary casting, which has been
       removed by this fix. (Bug #29807741, Bug #95210)

     * ResultSetMetaData.getTableName() returned null when no
       applicable results could be returned for a column.
       However, the JDBC documentation specified an empty string
       to be returned in that case. This fix makes the method
       behave as documented. The same correction has been made
       for getCatalogName() and getSchemaName(). (Bug #29452669,
       Bug #94585)

     * ResultSetImpl.getObject(), when autoboxing a value of a
       primitive type retrieved from a column, returned a
       non-null object when the retrieved value was null. (Bug
       #29446100, Bug #94533)

     * ResultSetImpl.getDouble() was very inefficient because it
       called FloatingPointBoundsEnforcer.createFromBigDecimal,
       which needlessly recreated BigDecimal objects for the
       fixed minimum and maximum bounds. With this fix, the
       objects BigDecimal.valueOf(min) and
       BigDecimal.valueOf(max) are cached after they are first
       created, thus avoiding their recreations. (Bug #29446059,
       Bug #94442)

     * Enabling logSlowQueries resulted in many unnecessary
       calls of LogUtils.findCallingClassAndMethod(). With this
       fix, LogUtils.findCallingClassAndMethod() is called only
       when profileSQL is true and even in that case, the number
       of calls are reduced to a minimal to avoid the excessive
       stack trace data the function used to generate. Thanks to
       Florian Agsteiner for contributing to the fix. (Bug
       #29277648, Bug #94101, Bug #17640628, Bug #70677)

     * Characters returned in a ResultSet were garbled when a
       server-side PreparedStatement was used, and the query
       involved concatenation of a number and a string with
       multi-byte characters. That was due to an issue with the
       number-to-string conversion involved, which has been
       corrected by this fix. (Bug #27453692)

     * Calling ProfilerEvent.pack() resulted in an
       ArrayIndexOutOfBoundsException. It was due to a
       mishandling of data types, which has been corrected by
       this fix. (Bug #11750577, Bug #41172)

Enjoy and thanks for the support!

On Behalf of MySQL/ORACLE RE Team
Sreedhar S

Working with SSH Tunneling and SSL PEM Certificates in Connector/NET

Dear MySQL Connector/NET community,

We are proud to announce that version 8.0.17 is introducing support for SSH tunneled connections through the classic MySQL protocol and X Protocol. SSH enables the creation of secure encrypted connections between the local and a remote computer allowing services or components to be accessed, MySQL Server in this case. With SSH tunneling, users can connect to a MySQL Server from behind a firewall when the server port is blocked. The server doesn’t require any additional configuration for this type of connection and continues to work as usual.

Users can also add an extra layer of security by making use of SSL over SSH connections, which brings us to the second announcement. Connector/NET previously included support for SSL connections via PFX certificates, which are exclusive to Windows. However, now the support for SSL PEM certificates is available starting with version 8.0.16. This allows Connector/NET to align with other MySQL products. More importantly, it increases the security of your applications, specially when you are working with .NET Core to expand your deployment to platforms beyond Windows.

Connection Options Overview

Those unfamiliar with Connector/NET and/or the differences between the classic MySQL protocol and the X Protocol, first need to be aware of the different ways to construct a connection/session object with which a connection to a MySQL Server can be established. There are 4 methods available:

  • Connection string: Available in the classic MySQL protocol and X Protocol. For the classic protocol an instance of the MySqlConnection class is created. In the constructor you provide a string containing key-value pairs which represent the connection options. For the X Protocol, call the MySQLX.GetSession method and provide the connection string in the constructor as well.
  • Using a builder object: Available in the classic MySQL protocol and X Protocol. An instance of the MySqlConnectionStringBuilder (classic protocol) or MySqlXConnectionStringBuilder (X Protocol) is created. You assign values to the mentioned object’s properties which resemble the connection options available in the connection string. Once the properties have been set, call the ToString method to get the connection string and simply pass it to the MySqlConnection object or MySQLX.GetSession method.
  • URI-like connection string: Available in the X Protocol only. A URI-like string is provided which starts with the mysqlx identifier. It supports the same connection options available for any other connection method.
  • Using anonymous objects: Available in the X Protocol only. An anonymous object is created having properties that match the name of the properties defined in the MySqlXConnectionStringBuilder class. The object is simply passed to the MySQLX.GetSession static method.

Refer to the Connection Examples section for code snippets on using each method.

Creating SSH Tunneled Connections

SSH Connection Options

New connection options have been introduced to support SSH tunneled connections. The connection options can either be specified in the connection string or by making use of an instance of the MySqlConnectionStringBuilder or MySqlXConnectionStringBuilder class. Find the new connection options and their details below:

Connection OptionDescriptionProperty NameConnection String Options
SSH Host NameThe IP address or name of the SSH server.SshHostNamesshHostName, ssh host name, ssh-host-name
SSH PortThe port of the SSH server.SshPortsshPort, ssh port, ssh-port
SSH User NameThe name of the SSH user to use to make a connection.SshUserNamesshUserName, ssh user name, ssh-user-name
SSH PasswordThe SSH password for the selected user.SshPasswordsshPassword, ssh password, ssh-password
SSH Key FileA path to the SSH key file including the filename and extension.SshKeyFilesshKeyFile, ssh key file, ssh-key-file
SSH Pass PhraseThe pass phrase of the SSH key file (if any).SshPassPhrasesshPassPhrase, ssh pass phrase, ssh-pass-phrase

Initiate an SSH Connection

First, the connection protocol being used must be TCP/IP, note that this is the default connection protocol in Connector/NET. Additional to that, the presence of the SSH User Name and either the SSH Password or SSH Key File options will notify Connector/NET of the user’s intent to make use of an SSH connection. Be sure to specify said options.

It is also necessary to specify the IP address or host name of the SSH Server being accessed. If no SSH Port is specified, the connection will default to the commonly known port 22.

An SSH Password or SSH Pass Phrase can also be provided in the event that they are required by the SSH User Name or SSH Key File connection options respectively.

Finally, set the Server connection option to localhost and Port to the port number the MySQL Server is running at in the SSH server.

Fallback for SSH Connections

Connector/NET includes a fallback mechanism when working with SSH connections. If values are provided to connect using both a SSH Key File and a SSH Password, fallback from the key file to the password will take place in the event that the former attempt fails. Note that the fallback is only enabled whenever the error is detected on the server side, this meaning that if the error is on the client side the user will instead get an exception prior to the connection attempts being made. Causes for errors on the client side can be an invalid key file or an incorrect pass phrase. 

Connection Examples

The following code examples will get you started on creating SSH Tunneled connections:

Example 1
// This is the most basic form of an SSH connection. MySQL port defaults to
// 3306 and SSH Port defaults to 22 since the values are not provided.
// Also, only a password is being used to authenticate to SSH server.
// Connection is being made using a MySqlConnectionStringBuilder object.
var builder = new MySqlConnectionStringBuilder();
builder.UserID = "myUser";
builder.Password = "test";
builder.Server = "localhost";
builder.SshHostName = "10.0.0.2";
builder.SshUserName = "mySshUser";
builder.SshPassword = "sshtest";
using (var connection = new MySqlConnection(builder.ConnectionString))
{
  connection.Open();
  connection.Close();
}
Example 2
// Now the MySQL and SSH ports are being provided.
// Again, only a password is being used to authenticate to SSH server.
// Connection is being made using a connection string.
using (var connection = new MySqlConnection("uid=myUser;password=test;server=localhost;port=3307;sshHostName=10.0.0.2;sshUserName=mySshUser;sshPassword=sshtest;sshPort=23"))
{
  connection.Open();
  connection.Close();
}
Example 3
// Here, a SSH key file with a pass phrase is provided. 
// Connection is being made using an anonymous object.
using (var connection = new MySqlConnection("uid=myUser;password=test;server=localhost;port=3307;sshHostName=10.0.0.2;sshUserName=mySshUser;sshKeyFile=C:\\keys\\myOpenSshKeyFile.ppk;sshPassPhrase=sshTest;sshPort=23"))
{
  connection.Open();
  connection.Close();
}
Example 4
// A pass phrase-less SSH key file and SSH password are both being provided.
// Given that the key file and pass phrase are valid a fallback to SSH 
// password will take place if authentication via SSH key file fails at 
// server side.
// place if a
var builder = new MySqlConnectionStringBuilder();
builder.UserID = "myUser";
builder.Password = "test";
builder.Server = "localhost";
builder.Port = 3307;
builder.SshHostName = "10.0.0.2";
builder.SshUserName = "mySshUser";
builder.SshKeyFile = @"C:\keys\noPassPhraseOpenSshKeyFile.ppk";
builder.SshPassword = "sshtest";
using (var connection = new MySqlConnection(builder.ConnectionString))
{
  connection.Open();
  connection.Close();
}
Example 5
// An X Protocol SSH connection using a URI-like connection string.
using (var session = MySQLX.GetSession("mysqlx://myUser:test@localhost:33060?sshHostName=10.0.0.2;sshUserName=mySshUser;sshPassword=sshTest"))
{
    session.Close();
}
Example 6
// An X Protocol SSH connection using an anonymous object along with
// setting an SSL Mode.
var sessionOptions = {
    UserID = "myUser",
    Password = "test",
    Server = "127.0.0.1",
    Port = 3307,
    SshHostName = "10.0.0.2",
    SshUserName = "mySshUser",
    SshKeyFile = @"C:\keys\myOpenSshKeyFile.ppk",
    SshPassPhrase = "sshtest",
    SslMode = MySqlSslMode.Required
  };
using (var session = MySQLX.GetSession(sessionOptions))
{
    session.Close();
}

Refer to the official documentation of Connector/NET for additional details on using SSH tunneling for your applications.

Creating SSL Connections with PEM Certificates

Connector/NET supports multiple levels of SSL encryption through the SSL Mode connection option. The list of allowed SSL encryption values are:

  • None: Do not use SSL.
  • Preferred: Use SSL, if server supports it.
  • Required: Always use SSL. Deny connection if server does not support SSL. Do not perform server certificate validation.
  • VerifyCA: Always use SSL. Validate server SSL certificate, but different host name mismatch.
  • VerifyFull: Always use SSL and perform full certificate validation.

The None, Preferred and Required SSL Modes don’t require the user to provide any certificates. Setting SSL Mode to VerifyCA requires that the SSL CA option is set. Setting SSL Mode to VerifyFull requires that SSL CA, SSL Cert and SSL Key are all set. Both VerifyCA and VerifyFull provide added security by performing various validations on the SSL certificates.

SSL Connection Options

Similar to how SSH tunneling introduced new connection options, a couple of connection options were added to support SSL PEM based connections. In past versions, the SSL CA and CertificateFile options could both be used to specify the path to a PFX certificate. Starting with version 8.0.16, the existing SSL CA option was extended to be used with SSL PEM connections as well. The full list of options relevant for SSL PEM connections can be found below:

Connection OptionDescriptionProperty NameConnection String Options
SSL CASpecifies the path to a certificate file in PEM format (.pem) that contains a list of trusted SSL certificate authorities (CA).SslCasslCa ,  ssl-Ca
SSL CertThe name of the SSL certificate file in PEM format to use for establishing an encrypted connection.SslCertsslCert ,  ssl-Cert
SSL KeyThe name of the SSL key file in PEM format to use for establishing an encrypted connection.SslKeysslKey , ssl-Key
SSL ModeSpecifies the level of security required for SSL connections.sslMode ,  ssl Mode ,  ssl-Mode

As previously mentioned, the SSL CA option is also supported in PFX based SSL connections. The extension of the file referenced in this connection option tells Connector/NET how to process the certificate and in the case of PEM based connections, it will notify Connector/NET to check the SSL Cert and SSL Key options.

Connection Examples

The following code examples will get you started on creating SSL PEM based connections:

Example 1
// Here only the SSL Ca option is required since the SSL Mode is set to VerifyCA.
// The connection is being made using a MySqlConnectionStringBuilder object.
var builder = new MySqlConnectionStringBuilder();
builder.UserID = "myUser";
builder.Password = "test";
builder.Server = "localhost";
builder.SslCa = "ca.pem";
builder.SslMode = MySqlSslMode.VerifyCA; 
using (var connection = new MySqlConnection(builder.ConnectionString))
{
  connection.Open();
  connection.Close();
}
Example 2
// All SSL options are required since the SSL Mode is set to VerifyFull.
// The connection is being made using a connection string.
using (var connection = new MySqlConnection("uid=myUser;password=test;server=localhost;port=3307;sslMode=VerifyFull;sslCa=ca.pem;sslCert=client-cert.pem;sslKey=client.key.pem"))
{
  connection.Open();
  connection.Close();
}

SSL PEM connections are also available for the X Protocol. SSL connection options are named similarly in the MySqlXConnectionStringBuilder class, anonymous object and URI-like connection string methods. Feel free to use the one that best adapts to your  needs. 

Additional Information and Useful Links

We really hope that you’ve enjoyed the time invested in getting to know these new features and that they prove useful and a step in the right direction to deliver more robust and secure MySQL based applications.

Your feedback is always welcome, so be sure to provide any comments or report any issues into our forums and dedicated bug page.

Finally, we leave you with the following links which will serve as reference to additional information about the topics discussed in this blog post and with Connector/NET in general. Thank you. 

MySQL for Excel 1.3.8 has been released

Dear MySQL users,

The MySQL Windows Experience Team is proud to announce the release of MySQL for Excel version 1.3.8. This is a maintenance release for 1.3.x. It can be used for production environments.

MySQL for Excel is an application plug-in enabling data analysts to very easily access and manipulate MySQL data within Microsoft Excel. It enables you to directly work with a MySQL database from within Microsoft Excel so you can easily do tasks such as:

  * Importing MySQL Data into Excel

  * Exporting Excel data directly into MySQL to a new or existing table

  * Editing MySQL data directly within Excel

MySQL for Excel is installed using the MySQL Installer for Windows.

The MySQL Installer comes in 2 versions

– Full (400 MB) which includes a complete set of MySQL products with
  their binaries included in the download.

– Web (18 MB – a network install) which will just pull the MySQL for
  Excel over the web and install it when run.

You can download MySQL Installer from our official Downloads page at
http://dev.mysql.com/downloads/installer/

The MySQL for Excel product can also be downloaded by using the product standalone installer found at this link http://dev.mysql.com/downloads/windows/excel/

Changes in MySQL for Excel 1.3.8 (2019-06-10, General Availability)

     * Functionality Added or Changed

     * Bugs Fixed

Functionality Added or Changed

     * Previously, 1000 (first rows of a MySQL table) was the
       value limit for previewing a small amount of data in
       Excel. However, setting the value to 300 or greater
       generated an exception and prevented additional editing
       operations. The upper threshold now is 100, instead of
       1000 (see Advanced Import Data Options, General Tab
       (https://dev.mysql.com/doc/mysql-for-excel/en/mysql-for-e
xcel-import-options-advanced.html#mysql-for-excel-import-
options-advanced-general)). (Bug #29745518)

     * A new global option, Tolerance for FLOAT and DOUBLE
       comparisons in WHERE clause, provides a way to edit data
       of these types that enables proper row-matching in the
       database when it is used together with optimistic updates
       (see Global Options, Edit Sessions Tab
       (https://dev.mysql.com/doc/mysql-for-excel/en/mysql-for-e
xcel-config-options.html#mysql-for-excel-global-options-e
dit-sessions)). (Bug #29179195, Bug #93824)

     * The Import Data operation adds digits to floating-point
       numbers. For example, instead of rendering a value such
       as 5.3 precisely from the database, the operation
       displays 5.0000019073486 after importing the data. This
       behavior affects FLOAT and DOUBLE data types, which
       adhere to the IEEE-754 standard and are stored as
       approximate values.
       A new option now provides a way to import floating-point
       numbers using the DECIMAL data type, which then stores
       and displays the exact value from the database (see
       Advanced Import Data Options, Formatting Tab
       (https://dev.mysql.com/doc/mysql-for-excel/en/mysql-for-e
xcel-import-options-advanced.html#mysql-for-excel-import-
advanced-format)). (Bug #26008777)

     * Support was added for encrypted connections in the form
       of SSL certificates and SSH tunneling, without the
       requirement of having intermediate proxy software to
       create the tunnel. Encrypted connections can be
       configured from the MySQL for Excel add-in directly or
       they can be configured with MySQL Workbench and then used
       to open a connection from the add-in. (Bug #18550080)

     * The Import Data operation for stored procedures now
       enables the selection of individual columns to be
       imported from each returned result set, which is similar
       to the way imported column data already works for table
       and view data. (Bug #16239029)

Bugs Fixed

     * The Export and Append Data actions for a cell with data
       in a worksheet were transferred unexpectedly to a cell
       without data on a second worksheet when the active focus
       was shifted to the second worksheet. (Bug #29839647)

     * A lack of contrast between onscreen message data and the
       background obscured the connection information when some
       themes (such as Dark Gray) were set on the host. This fix
       extends the selected theme colors to the MySQL for Excel
       add-in for the following versions of Excel: 2007, 2010,
       2013, 2016, 365, and 2019. (Bug #29826900)

     * When mappings were stored for Append Data operations, the
       add-in did not check for blank and duplicate stored
       mapping names. Now, validation ensures that all names are
       unique and that existing names are not overwritten
       without permission. (Bug #29826762)

     * Microsoft Excel prompted users to save workbooks that
       were unchanged. This fix alters the way metadata for
       connection information (used by Import and Edit Data
       operations) is created and stored, and ignores unrelated
       actions. (Bug #29625731)

     * When schema information was retrieved using a stored
       procedure, the operation was unable to find the
       mysql.proc system table. The operation now retrieves
       schema information from INFORMATION_SCHEMA tables.
       (Bug#29215137, Bug #93814)

     * With the option to create Excel relationships for
       imported tables enabled, an attempt to import a table
       (with related tables) generated an exception when the
       tables had circular references. This fix modifies the way
       relationships are created for Import Data operations for
       multiple tables, such that relationships among tables
       that could create a circular reference are not added to
       the Excel data model. (Bug #29050558)

     * The Edit Data operation returned an error message
       intermittently (value not suitable to be converted to a
       DateTime data type), even for tables without a DATETIME
       column. This fix updates the library used for internal
       connections to MySQL 8.0 server instances and the
       caching_sha2_password plugin. In addition, the updated
       library resolves an error in which fetched schema
       information for columns returns the rows in alphabetical
       order, instead of ordinal order. (Bug #29030866,
       Bug#93501)

     * Data imported to a worksheet could not be refreshed if
       the worksheet was renamed after the import operation. The
       add-in now inspects the connection information of
       imported tables to determine whether the associated
       worksheet name changed, and if so, it updates the
       connection metadata. Also, it removes the connection
       information for missing or deleted worksheets.
       (Bug#27441407, Bug #89387)

     * After editing, committing, and then refreshing the data
       from the database, subsequent commits were not recognized
       by the Edit Data operation. (Bug #27365464, Bug #87642)

     * An error was generated when an Edit Data operation
       involved changing the value of a date or time field. Now
       the value of each date or time field is wrapped with
       single quotes and the edits are saved to the database.
       (Bug #26301455, Bug #86723)

     * When an existing workbook was opened, a second (blank)
       workbook instance was also opened. (Bug #26245818,
       Bug#86633)

     * Some unsupported connection methods were shown as valid
       options to select. (Bug #26025950)

     * The Windows automatic scaling of visual elements did not
       work as expected when the operating system was configured
       to use a DPI value other than 100%. (Bug #23218058,
       Bug#81003)

Quick links:
MySQL for Excel documentation: http://dev.mysql.com/doc/en/mysql-for-excel.html.
Inside MySQL blog (NEW blog home): http://insidemysql.com/
MySQL on Windows blog (OLD blog home): http://blogs.oracle.com/MySQLOnWindows
MySQL for Excel forum: http://forums.mysql.com/list.php?172.
MySQL YouTube channel: http://www.youtube.com/user/MySQLChannel.

Enjoy and thanks for the support!
The MySQL on Windows team at Oracle.

MySQL Connectors 8.0.16 – Time to Catch up!

HI again! It’s been some time since I wrote and shared some of our updates with you. My intention back then was to give you an update with every release — if only all my plans worked out! However we’ve now had 3 releases since last I shared an update with you so it’s time that I updated everyone on what we’ve been working on.

All of our connectors (well except for ODBC) implement our new X DevAPI and so I’ll briefly go over these features and then break out some of the major connector specific features included.

X DevAPI

Handling of default schema

We had some inconsistencies regarding how a default schema given on a connection string was handled by some of our connectors. If a user gives a schema on the connection string — such as this – it was expected that myschema would be the default schema set on the session. This would mean that any SQL issued against that session would work against that schema. Not all connectors were handling this the correct way.

var session = mysqlx.getSession("mysqlx://localhost/myschema")

Various API standardization efforts

With many of our releases we’ll included various API standardizing efforts. We are always comparing our connectors and trying to make sure that the “core” API is essentially the same among all of them. When we see differences we schedule updates to correct those. Our goal is that if you understand the X DevAPI in one language then you can understand it in all the languages.

Prepared Statement support

With 8.0.16 we released prepared statement support when working with our X DevAPI/Document Store connectors. The first thing you may notice is that there isn’t any specific API for accessing prepared statements. This fits in with our general philosophy of letting the connectors “do the right thing” for the user rather than force the user to always write code. In this case the connector will notice that you are executing a “preparable” query more than once and will automatically make use of prepared statements for you. As of right now there is no way to configure or adjust this behavior. You should see a significant performance improvement from 8.0.15 to 8.0.16 when doing something like a loop of inserts or updates.

Connection Pooling Enhancements

An enhancement was included in 8.0.16 to improve our connection pooling by speeding how our connectors could “reset” the internal connection to the server. This is entirely an internal improvement. Developers will see this as a faster connection pool.

Connection Attributes

Connection attributes are a feature where you can pass in key value pairs when a connection is made. These values will be stored in the server performance schema and can be queried later as a way of “profiling” your application. Think of it simply as “tagging” your connections. These values can be given when the pool is created or when old style sessions are opened. Here is an example code snippet giving some connection attributes when opening an old-style session:

var session = mysqlx.getSession("mysqlx://localhost/schema
?connection-attributes=[key1=value1,key2=value2]")

Connector specific Features

Now I’d like to make a quick list of some of the larger changes we made to our connectors that are *not* connected to the X DevAPI. These are not all of the changes in our connectors. Each connector puts out a release announcement giving the entire change list with each release.

  • Connector/CPP
    • Support for MSVC 2017
    • Improved internal string handling
    • Improved our CMaked-based build system
  • Connector/Python
    • Removed support for Django versions less then 1.11
  • Connector/Net
    • Can now use SSL certificates in the PEM file format

As always, we really appreciate your willingness to work with our products and look forward to your feedback.

MySQL Connector/Python 8.0 – A year in

It’s been a year since MySQL 8.0 became GA and 8.0.16 has just been released. A good time to look at what happened with MySQL Connector/Python over the last few years.

pypi presence

When we created our connector we knew hat providing it via PyPI was important and we used PyPI as distribution channel. Later PEP 470 was published, which changed how packages are hosted and we introduced the C Extension, which required re-working the packaging. It took us a bit time to get all things right, but for a while we are now back on PyPI and you can get it not only from or downloads page, but also with a simple install using the pip tool:

After installation, which of course also works in a virtual environment, usage is just as it had been installed using the other packages, without hurdles as dependencies like protobuf are installed automatically:

Making the C extension default

Initially our Connector was written in 100% Python. This gives great portability to all platforms and all Python runtime implementations. However en- and decoding the network packages and doing all the related handling can take some time. To bring you the best performance we, some while ago, introduced the C Extension, which builds around libmysql and by using C can notably improve the performance. With 8.0 we took the big step and made it the default. Thus users now automatically benefit from the performance boost, without a change to the application. If however you want to use the pure python implementation just set the use_pure option to True and the old default is back. The pure Python version is still maintained for maximum compatibility and used automatically if the C extension can’t be loaded. If the C extension is available can be easily checked:

Introduction of X DevAPI and Document Store

A big change in all parts of MySQL 8.0 was the introduction of he MySQL Document Store, with a new network protocol and a new API. The MySQL Document Store allows easy access to JSON-style documents and CRUD access to tables directly from a more high-level API. Jesper gave a good introduction, so I won’t repeat it here. Of course you get the X DevAPI support as part of the PyPI package and of course the boost using the C extension is default as well.

EOL for MySQL Connector/Python 2

In January we have put the 2.1 series out of support to be able to to fully focus on 8.0. But don’t be afraid by the big jump in the version number. Version 8.0 is fully compatible. The only potential break is that we are defaulting now to the C extension as mentioned above. he only effect should be higher performance, but if that causes an issue you can go back to the pure Python version and please file a bug about why you need that.

More to come

Predictions are hard, especially about the future, but what I know for sue is, that there is a bright future for the MySQL Connector/Python. The main focus for the future is in three areas:

  • Making sure we give access to all the great current and upcoming features in the MySQL Server to Python users
  • Improve the X DevAPI to make writing new-style applications even more productive and simpler.
  • Improve integration into the Python ecosystem by continuing work on our Django integration and implementing different requests from the SQLAlchemy community.

Not really surprising goals, I guess. If you have any needs, please reach out to us!

MySQL Connector/ODBC 5.3.13 has been released

Dear MySQL users,

MySQL Connector/ODBC 5.3.13, a new version of the ODBC driver for the
MySQL database management system, has been released.

The available downloads include both a Unicode driver and an ANSI
driver based on the same modern codebase. Please select the driver
type you need based on the type of your application – Unicode or ANSI.
Server-side prepared statements are enabled by default. It is suitable
for use with any MySQL version from 5.6.

This is the sixth release of the MySQL ODBC driver conforming to the
ODBC 3.8 specification. It contains implementations of key 3.8
features, including self-identification as a ODBC 3.8 driver,
streaming of output parameters (supported for binary types only), and
support of the SQL_ATTR_RESET_CONNECTION connection attribute (for the
Unicode driver only).

The release is now available in source and binary form for a number of
platforms from our download pages at

http://dev.mysql.com/downloads/connector/odbc/5.3.html

For information on installing, please see the documentation at

http://dev.mysql.com/doc/connector-odbc/en/connector-odbc-installation.html

Changes in MySQL Connector/ODBC 5.3.13 (2019-04-29, General Availability)

Bugs Fixed

* Connector/ODBC 5.3 is now built with MySQL client library
5.7.26, which includes OpenSSL 1.0.2R. Issues fixed in
the new OpenSSL version are described at
http://www.openssl.org/news/vulnerabilities.html. (Bug
#29489006)

* An exception was emitted when fetching contents of a
BLOB/TEXT records after executing a statement as a
server-side prepared statement with a bound parameter.
The workaround is not using parameters or specifying
NO_SSPS=1 in the connection string; this allows the
driver to fetch the data. (Bug #29282638, Bug #29512548,
Bug #28790708, Bug #93895, Bug #94545, Bug #92078)

On Behalf of Oracle/MySQL Release Engineering Team,
Hery Ramilison

MySQL Shell 8.0.16 for MySQL Server 8.0 and 5.7 has been released

Dear MySQL users,

MySQL Shell 8.0.16 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle’s dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.16.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster, providing an
integrated solution for high availability and scalability using InnoDB
based MySQL databases, without requiring advanced MySQL expertise. For
more information about how to configure and work with MySQL InnoDB
cluster see

https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create “schema-less” JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language. For more
information about how to use MySQL Shell and the MySQL Document Store
support see

https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell’s SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.16, see the
“Generally Available (GA) Releases” tab at

http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

http://bugs.mysql.com/

Enjoy and thanks for the support!

 

On Behalf of Oracle/MySQL Release Engineering Team,
Nawaz Nazeer Ahamed

MySQL Connector/J 8.0.16 has been released

Dear MySQL users,

MySQL Connector/J Version 8.0.16 is the GA release of the 8.0
branch of MySQL Connector/J. It is suitable for use with MySQL Server
versions 8.0, 5.7 and 5.6. It supports the Java Database
Connectivity (JDBC) 4.2 API, and implements the X DevAPI.

This release includes the following new features and changes, also
described in more detail on

https://dev.mysql.com/doc/relnotes/connector-j/8.0/en/news-8-0-16.html

As always, we recommend that you check the “CHANGES” file in the
download archive to be aware of changes in behavior that might affect
your application.

To download MySQL Connector/J 8.0.16 GA, see the “Generally Available
(GA) Releases” tab at http://dev.mysql.com/downloads/connector/j/

Enjoy!

———————————————————————–

Changes in MySQL Connector/J 8.0.16 (2019-04-25, General Availability)

Functionality Added or Changed

* X DevAPI: Added BigInteger, BigDecimal, and Character as
supported classes whose instances can be passed to a
Table as Expression instances. Also made the error
message clearer when applications try to pass instances
of unsupported classes. (Bug #25650912)

* X DevAPI: Connector/J now supports the ability to send
connection attributes
(http://dev.mysql.com/doc/refman/8.0/en/performance-schema-connection-attribute-tables.html)
(key-value pairs that application programs can pass to the
server at connect time) for X Protocol connections.
Connector/J defines a default set of attributes
(http://dev.mysql.com/doc/refman/8.0/en/performance-schema-connection-attribute-tables.html#performance-schema-connection-attributes-cj),
which can be disabled or enabled. In addition,
applications can specify attributes to be
passed in addition to the default attributes. The default
behavior is to send the default attribute set. See the
description for the new configuration property
xdevapi.connect-attributes for details.
Note
The aggregate size of connection attribute data sent by a
client is limited by the value of the
performance_schema_session_connect_attrs_size server
variable. The total size of the data package should be
less than the value of the server variable, or the
attribute data will be truncated.

* X DevAPI: When using X DevAPI, performance for statements
that are executed repeatedly (two or more times) is
improved by using server-side prepared statements for the
second and subsequent executions. See Working with
Prepared Statements
(http://dev.mysql.com/doc/x-devapi-userguide/en/working-with-prepared-statements.html)
in the X DevAPI User Guide
(http://dev.mysql.com/doc/x-devapi-userguide/en/) for details.

* The version number has been removed from the name of the
Connector/J JAR archive within the RPM packages for
Connector/J. That makes upgrading Connector/J with RPM
packages easier. (Bug #29384853)

* The collation utf8mb4_zh_0900_as_cs has been added to the
CharsetMapping class. (Bug #29244101)

* The following third-party libraries have been removed
from the distribution bundles for Connector/J:

+ Google protobuf for Java (required for using X
DevAPI and for building Connector/J from source)

+ C3P0 (required for building Connector/J from source)

+ JBoss common JDBC wrapper (required for building
Connector/J from source)

+ Simple Logging Facade API (required for using the
logging capabilities provided by the default
implementation of org.slf4j.Logger.Slf4JLogger by
Connector/J, and for building Connector/J from
source)
Users who need those libraries have to obtain them on
their own. See Installing Connector/J from a Binary
Distribution
(http://dev.mysql.com/doc/connector-j/8.0/en/connector-j-binary-installation.html)
and Installing from Source
(http://dev.mysql.com/doc/connector-j/8.0/en/connector-j-installing-source.html)
for details.

Bugs Fixed

* X DevAPI: The method unquoteWorkaround() has been removed
from the ExprParser class, as the workaround is no longer
needed, and it actually produced wrong results in some
cases. (Bug #29257922)

* X DevAPI: Connector/J threw an error when a JSON document
contained only a field with an empty array as its value.
With this fix, Connector/J now takes that as a valid JSON
document. (Bug #28834959, Bug #92819)

* getByte() calls failed on table columns of the
BINARY data type. This was due to issues with string
conversion, which has been corrected with this fix. (Bug
#25650385)

* X DevAPI: Any statements sent after a failed procedure
call caused Connector/J to hang. This was because after
the failed call, Connector/J was not aware that the
result streamer had already been closed by the server.
With this fix, an error is thrown when the procedure call
fails, and the result streamer is nullified. (Bug
#22038729)

* X DevAPI: Unary negative and positive operators inside
expressions were parsed wrongly as binary minus and plus
operators. (Bug #21921956)

* Because the SHOW PROCESSLIST
(http://dev.mysql.com/doc/refman/8.0/en/show-processlist.html)
statement might cause the server to fail sometimes,
Connector/J now avoids using the statement, but queries
the performance scheme instead for the information it
needs. (Bug #29329326)

* Some unnecessary information has been removed from the
Connector/J log. (Bug #29318273)

* In the DatabaseMetaDataUsingInfoSchema interface, the
getProcedureColumns() and getFunctionColumns() methods
returned wrong results for the PRECISION column, and the
getColumns() and getVersionColumns() methods returned
wrong results for the COLUMN_SIZE column. The errors were
due to the wrong handling of the temporal type precision
by Connector/J, which has now been fixed. (Bug #29186870)

* For an SSL connection, after a client disconnected from a
server by calling Connection.close(), the TCP connection
remained in the TIME_WAIT
(http://dev.mysql.com/doc/refman/8.0/en/can-not-connect-to-server.html)
state on the server side. With this fix,
the connection remains in the TIME_WAIT
(http://dev.mysql.com/doc/refman/8.0/en/can-not-connect-to-server.html)
state on the client side instead, in most
cases. (Bug #29054329, Bug #93590)

* The function
LoadBalancedConnectionProxy.getGlobalBlacklist() always
returned an empty map, thus there was never a blacklist
for load-balanced connections. (Bug #28860051, Bug
#93007)

* The redundant file, changelog.gz, has been removed from
the Debian 9 package for Connector/J. The file repeated
the contents of the CHANGES.gz file. (Bug #27786499)

* Using getBytes() to retrieve TEXT data resulted in a
NumberFormatException. With this fix, the proper
exception (SQLDataException), is now thrown. (Bug
#27784363)

* A changeUser() call failed with a java.io.IOException
when the configuration property enablePacketDebug was set
to true for a connection. (Bug #25642021)

* bindings.getBoolean() always returned false. It was due
to a mishandling of data types, which has been corrected
with this fix. (Bug #22931700)

Enjoy and thanks for the support!

On Behalf of MySQL/ORACLE RE Team
Sreedhar S

MySQL Connector/ODBC 8.0.16 has been released

Dear MySQL users,

MySQL Connector/ODBC 8.0.16 is a new version in the MySQL Connector/ODBC 8.0 series,
the ODBC driver for the MySQL Server.

The available downloads include both a Unicode driver and an ANSI driver based on the
same modern codebase. Please select the driver type you need based on the type of your
application – Unicode or ANSI. Server-side prepared statements are enabled by default.
It is suitable for use with the latest MySQL server version 8.0.

This release of the MySQL ODBC driver is conforming to the ODBC 3.8 specification.
It contains implementations of key 3.8 features, including self-identification
as a ODBC 3.8 driver, streaming of output parameters (supported for binary types
only), and support of the SQL_ATTR_RESET_CONNECTION connection attribute (for the
Unicode driver only).

The release is now available in source and binary form for a number of platforms
from our download pages at

https://dev.mysql.com/downloads/connector/odbc/

For information on installing, please see the documentation at

https://dev.mysql.com/doc/connector-odbc/en/connector-odbc-installation.html

Enjoy and thanks for the support!

==================================================

Changes in MySQL Connector/ODBC 8.0.16 (2019-04-25, General Availability)

Bugs Fixed

* Connector/ODBC 8.0 is now built with OpenSSL 1.0.2R.
Issues fixed in the new OpenSSL version are described at
http://www.openssl.org/news/vulnerabilities.html. (Bug
#29538143)

* An exception was emitted when fetching contents of a
BLOB/TEXT records after executing a statement as a
server-side prepared statement with a bound parameter.
The workaround is not using parameters or specifying
NO_SSPS=1 in the connection string; this allows the
driver to fetch the data. (Bug #29282638, Bug #29512548,
Bug #28790708, Bug #93895, Bug #94545, Bug #92078)

On Behalf of Oracle/MySQL Release Engineering Team,
Hery Ramilison

MySQL Connector/NET 8.0.16 has been released

Dear MySQL users,

MySQL Connector/NET 8.0.16 is the fourth version to support
Entity Framework Core 2.1 and the sixth general availability release
of MySQL Connector/NET to add support for the new X DevAPI, which
enables application developers to write code that combines the
strengths of the relational and document models using a modern,
NoSQL-like syntax that does not assume previous experience writing traditional SQL.

To learn more about how to write applications using the X DevAPI, see
http://dev.mysql.com/doc/x-devapi-userguide/en/index.html. For more
information about how the X DevAPI is implemented in Connector/NET, see
http://dev.mysql.com/doc/dev/connector-net. NuGet packages provide functionality at a project level. To get the
full set of features available in Connector/NET such as availability
in the GAC, integration with Visual Studio’s Entity Framework Designer
and integration with MySQL for Visual Studio, installation through
the MySQL Installer or the stand-alone MSI is required.

Please note that the X DevAPI requires at least MySQL Server version
8.0 or higher with the X Plugin enabled. For general documentation
about how to get started using MySQL as a document store, see
http://dev.mysql.com/doc/refman/8.0/en/document-store.html.

To download MySQL Connector/NET 8.0.16, see
http://dev.mysql.com/downloads/connector/net/

Installation instructions can be found at
https://dev.mysql.com/doc/connector-net/en/connector-net-installation.html

Changes in MySQL Connector/NET 8.0.16 ( 2019-04-25, General Availability )

* Functionality Added or Changed

* Bugs Fixed

Functionality Added or Changed

* Document Store: Support was added for the -> operator to
be used with JSON document paths in relational
statements. For example:
table.Select().Where("additionalinfo->$.hobbies = 'Reading'");

(Bug #29347028)

* Document Store: The performance for statements that are
executed repeatedly (two or more times) is improved by
using server-side prepared statements for the second and
subsequent executions. This happens internally;
applications need take no action and API behavior should
be the same as previously. For statements that change,
repreparation occurs as needed. Providing different data
values or different OFFSET or LIMIT clause values does
not count as a change. Instead, the new values are passed
to a new invocation of the previously prepared statement.

* Document Store: Connector/NET now supports the ability to
send connection attributes (key-value pairs that
application programs can pass to the server at connect
time). Connector/NET defines a default set of attributes,
which can be disabled or enabled. In addition,
applications can specify attributes to be passed together
with the default attributes. The default behavior is to
send the default attribute set.
The aggregate size of connection attribute data sent by a
client is limited by the value of the
performance_schema_session_connect_attrs_size server
variable. The total size of the data package should be
less than the value of the server variable. For X DevAPI
applications, specify connection attributes as a
connection-attributes parameter in a connection string.
For usage information, see Options for X Protocol Only
(http://dev.mysql.com/doc/connector-net/en/connector-net-8-0-connection-options.html#connector-net-8-0-connection-options-xprotocol).
For general information about connection attributes, see
Performance Schema Connection Attribute Tables
(http://dev.mysql.com/doc/refman/8.0/en/performance-schema-connection-attribute-tables.html).

* Document Store: Connector/NET now has improved support
for resetting sessions in connection pools. Returning a
session to the pool drops session-related objects such as
temporary tables, session variables, and transactions,
but the connection remains open and authenticated so that
reauthentication is not required when the session is
reused.

* Connector/NET applications now can use certificates in
PEM format to validate SSL connections in addition to the
native PFX format (see Tutorial: Using SSL with Connector/NET
(http://dev.mysql.com/doc/connector-net/en/connector-net-tutorials-ssl.html)).
PEM support applies to both classic MySQL protocol
and X Protocol connections.

Bugs Fixed

* Document Store: All methods able to execute a statement
were unable to execute the same statement a second time.
Now, the values and binding parameters remain available
after the method is executed and string parameters are no
longer converted to numbers. Both changes enable a
follow-on execution to reuse the previous parameters.
(Bug #29249857, Bug #29304767)

* An exception was generated when the MySqlDbType
enumeration was given an explicit value and then passed
as a parameter to the MySqlCommand.Prepare method. (Bug
#28834253, Bug #92912)

* Validation was added to ensure that when a column is of
type TIME and the value is 00:00:00, it takes the value
instead of setting NULL. (Bug #28383726, Bug #91752)

On Behalf of MySQL Release Engineering Team,
Surabhi Bhat