You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@grails.apache.org by jd...@apache.org on 2026/10/22 19:27:45 UTC
(grails-core) branch feat/sql-tail-command updated: Redact Spanner statements as GoogleSQL in sql-tail and read a truncated capture again
This is an automated email from the ASF dual-hosted git repository.
jdaugherty pushed a commit to branch feat/sql-tail-command
in repository https://gitbox.apache.org/repos/asf/grails-core.git
The following commit(s) were added to refs/heads/feat/sql-tail-command by this push:
new afaaabb58a Redact Spanner statements as GoogleSQL in sql-tail and read a truncated capture again
afaaabb58a is described below
commit afaaabb58acb2a5021b98827b98586762cfc4c0a
Author: James Daugherty <jd...@jdresources.net>
AuthorDate: Thu Oct 22 15:27:23 2026 -0400
Redact Spanner statements as GoogleSQL in sql-tail and read a truncated capture again
Cloud Spanner's dialect quotes identifiers with backticks, so its statements were redacted as
MySQL, which has no triple-quoted string: `select """a"example-secret"b"""` was captured as
`select ?example-secret?`, with part of the value. GoogleSQL is now a syntax of its own: single and
double quotes delimit strings, a string that opens with its quote written three times holds any
quote and ends at the next three, a backslash always escapes the character after it, a quote
written twice is two strings, backticks delimit identifiers, and # starts a comment. Hibernate 7's
Spanner dialect is redacted in it.
The inspectors no longer take every backtick-quoting dialect for MySQL: only MySQL's dialect and
its subclasses are, so a dialect Grails does not know, such as the Spanner dialect for Hibernate 5,
is read in every syntax, GoogleSQL among them, and a statement that triple-quotes a string, or
doubles a quote inside one, is withheld there.
The command's reader measured the capture and then read that much with readFully, so an
application that truncated the file in between ended the command with an EOFException. The reader
now reads what the file holds, and a read that comes up short, or finds the file gone, is a
restart: the new run is read from its start in the same poll.
---
.../sqltail/SqlTailStatementInspector.java | 14 ++--
.../sqltail/SqlTailStatementInspectorSpec.groovy | 19 +++++
.../sqltail/SqlTailStatementInspector.java | 18 ++--
.../sqltail/SqlTailStatementInspectorSpec.groovy | 21 ++++-
.../gorm/jdbc/sqltail/cli/SqlTailFileReader.groovy | 94 ++++++++++++++-------
.../gorm/jdbc/sqltail/SqlTailRedactor.java | 96 +++++++++++++++++-----
.../jdbc/sqltail/cli/SqlTailFileReaderSpec.groovy | 65 +++++++++++++++
.../gorm/jdbc/sqltail/SqlTailCaptureSpec.groovy | 14 ++++
.../gorm/jdbc/sqltail/SqlTailRedactorSpec.groovy | 41 +++++++++
.../src/en/guide/conf/dataSource/sqlTail.adoc | 13 +--
10 files changed, 327 insertions(+), 68 deletions(-)
diff --git a/grails-data-hibernate5/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java b/grails-data-hibernate5/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java
index 842383c4ee..5b3a1fe365 100644
--- a/grails-data-hibernate5/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java
+++ b/grails-data-hibernate5/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java
@@ -33,6 +33,7 @@ import org.hibernate.dialect.DB2Dialect;
import org.hibernate.dialect.Dialect;
import org.hibernate.dialect.H2Dialect;
import org.hibernate.dialect.HSQLDialect;
+import org.hibernate.dialect.MySQLDialect;
import org.hibernate.dialect.Oracle8iDialect;
import org.hibernate.dialect.Oracle9Dialect;
import org.hibernate.dialect.PostgreSQL81Dialect;
@@ -200,11 +201,12 @@ public class SqlTailStatementInspector implements StatementInspector, SessionFac
/**
* The syntax the statements of a dialect are redacted in. A Transact-SQL dialect, SQL Server's or
* any of Sybase's, reads double-quoted text as a string when {@code QUOTED_IDENTIFIER} is off, as
- * it is by default on Sybase, and a dialect that quotes identifiers with backticks reads it as a
- * string as MySQL does. Only the dialects of databases known to read double-quoted text as an
- * identifier are redacted as standard SQL. Other dialects that quote identifiers with double
- * quotes, such as those of SQLite and Informix, can still read double-quoted text as a string, so
- * their statements are redacted as for a database that is not known.
+ * it is by default on Sybase; MySQL's and MariaDB's read it as a string. Only the dialects of
+ * databases known to read double-quoted text as an identifier are redacted as standard SQL. Any
+ * other dialect, such as SQLite's or Informix's, which quote identifiers with double quotes but can
+ * read double-quoted text as a string, or one that quotes identifiers with backticks and is not
+ * Hibernate's, such as the Cloud Spanner dialect, whose GoogleSQL strings can be triple-quoted, is
+ * redacted as for a database that is not known.
*
* @param dialect the dialect of a session factory
* @return the syntax its statements are redacted in
@@ -213,7 +215,7 @@ public class SqlTailStatementInspector implements StatementInspector, SessionFac
if (dialect instanceof SQLServerDialect || dialect instanceof SybaseDialect) {
return SqlTailRedactor.Syntax.TRANSACT_SQL;
}
- if (dialect.openQuote() == '`') {
+ if (dialect instanceof MySQLDialect) {
return SqlTailRedactor.Syntax.MYSQL;
}
for (Class<? extends Dialect> standard : STANDARD_DIALECTS) {
diff --git a/grails-data-hibernate5/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy b/grails-data-hibernate5/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy
index e4908bdd7b..3d3821f454 100644
--- a/grails-data-hibernate5/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy
+++ b/grails-data-hibernate5/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy
@@ -33,6 +33,7 @@ import org.hibernate.SessionFactoryObserver
import org.hibernate.dialect.CockroachDB201Dialect
import org.hibernate.dialect.DB2Dialect
import org.hibernate.dialect.DerbyTenSevenDialect
+import org.hibernate.dialect.Dialect
import org.hibernate.dialect.H2Dialect
import org.hibernate.dialect.HANAColumnStoreDialect
import org.hibernate.dialect.HSQLDialect
@@ -151,6 +152,7 @@ class SqlTailStatementInspectorSpec extends Specification {
new SybaseAnywhereDialect() || TRANSACT_SQL
new InformixDialect() || UNKNOWN
new TeradataDialect() || UNKNOWN
+ new SqlTailSpannerDialect() || UNKNOWN
}
void "every data source of the application records to the one capture"() {
@@ -380,3 +382,20 @@ class SqlTailRecordingObserver implements SessionFactoryObserver {
events << 'closed'
}
}
+
+/**
+ * A dialect that is not Hibernate's, such as the Cloud Spanner dialect for Hibernate 5, which quotes
+ * identifiers with backticks: its GoogleSQL strings can be triple-quoted, which is not read as MySQL.
+ */
+class SqlTailSpannerDialect extends Dialect {
+
+ @Override
+ char openQuote() {
+ '`' as char
+ }
+
+ @Override
+ char closeQuote() {
+ '`' as char
+ }
+}
diff --git a/grails-data-hibernate7/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java b/grails-data-hibernate7/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java
index a37613ea32..0e25ee521d 100644
--- a/grails-data-hibernate7/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java
+++ b/grails-data-hibernate7/core/src/main/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspector.java
@@ -34,8 +34,10 @@ import org.hibernate.dialect.Dialect;
import org.hibernate.dialect.H2Dialect;
import org.hibernate.dialect.HANADialect;
import org.hibernate.dialect.HSQLDialect;
+import org.hibernate.dialect.MySQLDialect;
import org.hibernate.dialect.OracleDialect;
import org.hibernate.dialect.PostgreSQLDialect;
+import org.hibernate.dialect.SpannerDialect;
import org.hibernate.engine.spi.SessionFactoryImplementor;
import org.hibernate.resource.jdbc.spi.StatementInspector;
@@ -194,11 +196,12 @@ public class SqlTailStatementInspector implements StatementInspector, SessionFac
/**
* The syntax the statements of a dialect are redacted in. A Transact-SQL dialect, SQL Server's or
* any of Sybase's, reads double-quoted text as a string when {@code QUOTED_IDENTIFIER} is off, as
- * it is by default on Sybase, and a dialect that quotes identifiers with backticks reads it as a
- * string as MySQL does. Only the dialects of databases known to read double-quoted text as an
- * identifier are redacted as standard SQL. Other dialects that quote identifiers with double
- * quotes, such as those of SQLite and Informix, can still read double-quoted text as a string, so
- * their statements are redacted as for a database that is not known.
+ * it is by default on Sybase; Cloud Spanner's reads it as a string and can triple-quote one; MySQL's
+ * and MariaDB's read it as a string. Only the dialects of databases known to read double-quoted
+ * text as an identifier are redacted as standard SQL. Any other dialect, such as SQLite's or
+ * Informix's, which quote identifiers with double quotes but can read double-quoted text as a
+ * string, or a community dialect that quotes identifiers with backticks, is redacted as for a
+ * database that is not known.
*
* @param dialect the dialect of a session factory
* @return the syntax its statements are redacted in
@@ -207,7 +210,10 @@ public class SqlTailStatementInspector implements StatementInspector, SessionFac
if (dialect instanceof AbstractTransactSQLDialect) {
return SqlTailRedactor.Syntax.TRANSACT_SQL;
}
- if (dialect.openQuote() == '`') {
+ if (dialect instanceof SpannerDialect) {
+ return SqlTailRedactor.Syntax.GOOGLE_SQL;
+ }
+ if (dialect instanceof MySQLDialect) {
return SqlTailRedactor.Syntax.MYSQL;
}
for (Class<? extends Dialect> standard : STANDARD_DIALECTS) {
diff --git a/grails-data-hibernate7/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy b/grails-data-hibernate7/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy
index 5f56f1fa3b..8808ee7818 100644
--- a/grails-data-hibernate7/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy
+++ b/grails-data-hibernate7/core/src/test/groovy/org/grails/orm/hibernate/sqltail/SqlTailStatementInspectorSpec.groovy
@@ -48,6 +48,7 @@ import org.hibernate.resource.jdbc.spi.StatementInspector
import spock.lang.Specification
import spock.lang.TempDir
+import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.GOOGLE_SQL
import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.MYSQL
import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.STANDARD
import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.TRANSACT_SQL
@@ -141,10 +142,11 @@ class SqlTailStatementInspectorSpec extends Specification {
new HANADialect() || STANDARD
new MySQLDialect() || MYSQL
new MariaDBDialect() || MYSQL
- new SpannerDialect() || MYSQL
+ new SpannerDialect() || GOOGLE_SQL
new SQLServerDialect() || TRANSACT_SQL
new SybaseASEDialect() || TRANSACT_SQL
new SqlTailCommunityDialect() || UNKNOWN
+ new SqlTailBacktickDialect() || UNKNOWN
}
void "every data source of the application records to the one capture"() {
@@ -380,3 +382,20 @@ class SqlTailRecordingObserver implements SessionFactoryObserver {
*/
class SqlTailCommunityDialect extends Dialect {
}
+
+/**
+ * A dialect GORM does not know that quotes identifiers with backticks, as a community dialect for
+ * BigQuery would: its database may read strings as MySQL does, or as GoogleSQL does.
+ */
+class SqlTailBacktickDialect extends Dialect {
+
+ @Override
+ char openQuote() {
+ '`' as char
+ }
+
+ @Override
+ char closeQuote() {
+ '`' as char
+ }
+}
diff --git a/grails-datamapping-core/src/cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReader.groovy b/grails-datamapping-core/src/cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReader.groovy
index 30f552d1ef..5f79d4b4dc 100644
--- a/grails-datamapping-core/src/cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReader.groovy
+++ b/grails-datamapping-core/src/cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReader.groovy
@@ -30,7 +30,9 @@ import org.grails.datastore.gorm.jdbc.sqltail.SqlTailRecord
* the application is still writing is never read in half. An application that starts again
* truncates the file and names its new run in the first line; when that line names another run,
* or the file shrinks, reading starts over from the beginning. The run tells a restart apart from
- * a file that grew past where the reader was before it was next polled.
+ * a file that grew past where the reader was before it was next polled. A file truncated between
+ * being measured and being read, so that the read comes up short, is a restart too: it is read again
+ * from the beginning in the same poll.
*
* @since 8.1
*/
@@ -62,34 +64,31 @@ class SqlTailFileReader implements Closeable {
*/
List<SqlTailRecord> poll() {
restarted = false
- long length = file.length()
- String current = length > 0 ? readRun() : null
- if (length < position || (current != null && run != null && current != run)) {
+ // a read that comes up short is retried once, from the start; the next poll takes it from there
+ for (int attempt = 0; attempt < 2; attempt++) {
+ long length = file.length()
+ String current = length > 0 ? readRun() : null
+ if (length < position || (current != null && run != null && current != run)) {
+ position = 0
+ restarted = true
+ }
+ if (current != null || restarted) {
+ // a file truncated before its header was written names its run on a later poll
+ run = current
+ }
+ if (length <= position) {
+ return []
+ }
+ byte[] bytes = readFrom(position, length)
+ if (bytes.length == length - position) {
+ return recordsIn(bytes)
+ }
+ // the application truncated the file after it was measured: it has started again
position = 0
restarted = true
+ run = null
}
- if (current != null || restarted) {
- // a file truncated before its header was written names its run on a later poll
- run = current
- }
- if (length <= position) {
- return []
- }
- byte[] bytes = readFrom(position, length)
- int consumed = lastLineEnd(bytes)
- if (consumed == 0) {
- return []
- }
- position += consumed
- String text = new String(bytes, 0, consumed, StandardCharsets.UTF_8)
- List<SqlTailRecord> records = []
- for (String line : text.split('\n')) {
- SqlTailRecord record = SqlTailRecord.parse(line)
- if (record != null) {
- records.add(record)
- }
- }
- records
+ []
}
/**
@@ -118,6 +117,26 @@ class SqlTailFileReader implements Closeable {
// the file is opened and closed per poll
}
+ /**
+ * Takes the complete lines out of what was read, and leaves a line still being written for the next poll.
+ */
+ private List<SqlTailRecord> recordsIn(byte[] bytes) {
+ int consumed = lastLineEnd(bytes)
+ if (consumed == 0) {
+ return []
+ }
+ position += consumed
+ String text = new String(bytes, 0, consumed, StandardCharsets.UTF_8)
+ List<SqlTailRecord> records = []
+ for (String line : text.split('\n')) {
+ SqlTailRecord record = SqlTailRecord.parse(line)
+ if (record != null) {
+ records.add(record)
+ }
+ }
+ records
+ }
+
private static int lastLineEnd(byte[] bytes) {
for (int i = bytes.length - 1; i >= 0; i--) {
if (bytes[i] == NEWLINE) {
@@ -137,12 +156,29 @@ class SqlTailFileReader implements Closeable {
null
}
+ /**
+ * @return the bytes between the offsets, or fewer when the file ends before the second one or is
+ * gone: it was truncated, or removed, after it was measured
+ */
private byte[] readFrom(long from, long to) {
- RandomAccessFile input = new RandomAccessFile(file, 'r')
+ byte[] bytes = new byte[(int) Math.max(0L, to - from)]
+ RandomAccessFile input
+ try {
+ input = new RandomAccessFile(file, 'r')
+ }
+ catch (FileNotFoundException ignored) {
+ return new byte[0]
+ }
try {
input.seek(from)
- byte[] bytes = new byte[(int) Math.max(0L, Math.min(to, input.length()) - from)]
- input.readFully(bytes)
+ int read = 0
+ while (read < bytes.length) {
+ int count = input.read(bytes, read, bytes.length - read)
+ if (count < 0) {
+ return Arrays.copyOf(bytes, read)
+ }
+ read += count
+ }
bytes
}
finally {
diff --git a/grails-datamapping-core/src/main/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactor.java b/grails-datamapping-core/src/main/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactor.java
index e666b124ca..798e10f33a 100644
--- a/grails-datamapping-core/src/main/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactor.java
+++ b/grails-datamapping-core/src/main/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactor.java
@@ -28,11 +28,12 @@ import java.util.List;
* capture file. Only the recorded copy is redacted: the statement the database runs is not changed.
*
* <p>Databases disagree on parts of the lexical syntax: whether a backslash escapes a quote, whether
- * comments nest, whether double quotes delimit an identifier or a string, and whether {@code #}
- * starts a comment. A statement is redacted under every reading its {@link Syntax} allows, and is
- * recorded only when all of them give the same text. When they do not, or when a literal or a
- * comment never ends, the statement is withheld: {@link #WITHHELD} is recorded in its place, never
- * the statement.</p>
+ * comments nest, whether double quotes delimit an identifier or a string, whether a quote written
+ * twice escapes it or ends one string and starts another, whether a quote written three times
+ * opens a string that quotes can appear in, and whether {@code #} starts a comment. A statement is
+ * redacted under every reading its {@link Syntax} allows, and is recorded only when all of them give
+ * the same text. When they do not, or when a literal or a comment never ends, the statement is
+ * withheld: {@link #WITHHELD} is recorded in its place, never the statement.</p>
*
* @since 8.1
*/
@@ -69,7 +70,15 @@ public final class SqlTailRedactor {
* Microsoft SQL Server and Sybase: square brackets delimit identifiers, comments nest, and double
* quotes delimit a string when {@code QUOTED_IDENTIFIER} is off, as it is by default on Sybase.
*/
- TRANSACT_SQL
+ TRANSACT_SQL,
+
+ /**
+ * GoogleSQL, the dialect of Cloud Spanner: backticks delimit identifiers, single and double quotes
+ * delimit strings, a string that opens with its quote written three times ends only at the quote
+ * written three times again and holds any other quote, a backslash always escapes the character
+ * after it, a quote written twice is two strings, and {@code #} starts a comment.
+ */
+ GOOGLE_SQL
}
private SqlTailRedactor() {
@@ -121,15 +130,20 @@ public final class SqlTailRedactor {
if (syntax == Syntax.TRANSACT_SQL || syntax == Syntax.UNKNOWN) {
readings.add(new Reading(Syntax.TRANSACT_SQL, false, true));
}
+ if (syntax == Syntax.GOOGLE_SQL || syntax == Syntax.UNKNOWN) {
+ readings.add(new Reading(Syntax.GOOGLE_SQL, true, false));
+ }
return readings;
}
/**
- * One way of reading a statement. Only MySQL delimits identifiers with backticks, starts a comment
- * with {@code #}, and needs whitespace after {@code --}; only Transact-SQL delimits identifiers with
- * square brackets. Outside standard SQL, double quotes can delimit a string: in MySQL unless
- * {@code ANSI_QUOTES} is set, and in Transact-SQL when {@code QUOTED_IDENTIFIER} is off. Either way the
- * quoted run ends at the same place, so it is replaced as a value.
+ * One way of reading a statement. MySQL and GoogleSQL delimit identifiers with backticks and start a
+ * comment with {@code #}; only MySQL needs whitespace after {@code --}; only Transact-SQL delimits
+ * identifiers with square brackets. Outside standard SQL, double quotes can delimit a string: in
+ * MySQL unless {@code ANSI_QUOTES} is set, in Transact-SQL when {@code QUOTED_IDENTIFIER} is off, and
+ * in GoogleSQL always. Either way the quoted run ends at the same place, so it is replaced as a
+ * value. A GoogleSQL string is read on its own terms: a quote written twice ends it, a backslash
+ * escapes, and a quote written three times opens a string that ends only at the next three.
*
* @param syntax the syntax read in, never {@link Syntax#UNKNOWN}
* @param backslashEscapes whether a backslash escapes the character after it in a string
@@ -142,6 +156,7 @@ public final class SqlTailRedactor {
*/
String redact(String sql) {
boolean mysql = syntax == Syntax.MYSQL;
+ boolean googleSql = syntax == Syntax.GOOGLE_SQL;
boolean doubleQuotedValue = syntax != Syntax.STANDARD;
int length = sql.length();
StringBuilder out = new StringBuilder(length);
@@ -152,24 +167,28 @@ public final class SqlTailRedactor {
char previous = i > 0 ? sql.charAt(i - 1) : 0;
int dollarTag = c == '$' && !isIdentifierPart(previous) ? dollarTagLength(sql, i) : 0;
int end;
- if (c == '\'') {
+ if (googleSql && (c == '\'' || c == '"')) {
+ end = endOfGoogleSqlString(sql, i);
+ out.append(PLACEHOLDER);
+ }
+ else if (c == '\'') {
end = endOfQuotedLiteral(sql, i);
out.append(PLACEHOLDER);
}
else if (c == '"') {
- end = endOfQuoted(sql, i, '"', mysql && backslashEscapes);
+ end = endOfQuoted(sql, i, '"', mysql && backslashEscapes, true);
if (end >= 0) {
out.append(doubleQuotedValue ? PLACEHOLDER : sql.substring(i, end));
}
}
- else if (c == '`' && mysql) {
- end = endOfQuoted(sql, i, '`', false);
+ else if (c == '`' && (mysql || googleSql)) {
+ end = endOfQuoted(sql, i, '`', googleSql, mysql);
if (end >= 0) {
out.append(sql, i, end);
}
}
else if (c == '[' && syntax == Syntax.TRANSACT_SQL) {
- end = endOfQuoted(sql, i, ']', false);
+ end = endOfQuoted(sql, i, ']', false, true);
if (end >= 0) {
out.append(sql, i, end);
}
@@ -179,7 +198,7 @@ public final class SqlTailRedactor {
out.append(' ');
}
else if ((c == '-' && next == '-' && (!mysql || i + 2 >= length || Character.isWhitespace(sql.charAt(i + 2)))) ||
- (c == '#' && mysql)) {
+ (c == '#' && (mysql || googleSql))) {
end = endOfLine(sql, i);
out.append(' ');
}
@@ -219,7 +238,7 @@ public final class SqlTailRedactor {
return close < 0 ? -1 : close + 2;
}
boolean escapeString = (prefix == 'e' || prefix == 'E') && standalonePrefix;
- return endOfQuoted(sql, start, '\'', backslashEscapes || escapeString);
+ return endOfQuoted(sql, start, '\'', backslashEscapes || escapeString, true);
}
private int endOfBlockComment(String sql, int start) {
@@ -248,11 +267,13 @@ public final class SqlTailRedactor {
}
/**
- * Ends a quoted run, in which the closing quote written twice stands for itself.
+ * Ends a quoted run.
*
+ * @param backslashEscapes whether a backslash escapes the character after it
+ * @param doubledClose whether the closing quote written twice stands for itself
* @return the index after the closing quote, or -1 when there is none
*/
- private static int endOfQuoted(String sql, int start, char close, boolean backslashEscapes) {
+ private static int endOfQuoted(String sql, int start, char close, boolean backslashEscapes, boolean doubledClose) {
int i = start + 1;
while (i < sql.length()) {
char c = sql.charAt(i);
@@ -260,7 +281,7 @@ public final class SqlTailRedactor {
i += 2;
}
else if (c == close) {
- if (i + 1 < sql.length() && sql.charAt(i + 1) == close) {
+ if (doubledClose && i + 1 < sql.length() && sql.charAt(i + 1) == close) {
i += 2;
}
else {
@@ -274,6 +295,39 @@ public final class SqlTailRedactor {
return -1;
}
+ /**
+ * Ends a GoogleSQL string, which a backslash escapes in and a quote written twice does not: that
+ * is an empty string, or the end of one string and the start of the next. A string that opens
+ * with its quote written three times is triple-quoted: it holds any quote, and ends at its quote
+ * written three times again.
+ *
+ * @return the index after the closing quote, or -1 when there is none
+ */
+ private static int endOfGoogleSqlString(String sql, int start) {
+ char quote = sql.charAt(start);
+ if (start + 2 < sql.length() && sql.charAt(start + 1) == quote && sql.charAt(start + 2) == quote) {
+ return endOfTripleQuoted(sql, start, quote);
+ }
+ return endOfQuoted(sql, start, quote, true, false);
+ }
+
+ private static int endOfTripleQuoted(String sql, int start, char quote) {
+ int i = start + 3;
+ while (i < sql.length()) {
+ char c = sql.charAt(i);
+ if (c == '\\') {
+ i += 2;
+ }
+ else if (c == quote && i + 2 < sql.length() && sql.charAt(i + 1) == quote && sql.charAt(i + 2) == quote) {
+ return i + 3;
+ }
+ else {
+ i++;
+ }
+ }
+ return -1;
+ }
+
private static int endOfLine(String sql, int start) {
int i = start;
while (i < sql.length() && sql.charAt(i) != '\n' && sql.charAt(i) != '\r') {
diff --git a/grails-datamapping-core/src/test-cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReaderSpec.groovy b/grails-datamapping-core/src/test-cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReaderSpec.groovy
index d82e7200c6..16aae76248 100644
--- a/grails-datamapping-core/src/test-cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReaderSpec.groovy
+++ b/grails-datamapping-core/src/test-cli/groovy/org/grails/datastore/gorm/jdbc/sqltail/cli/SqlTailFileReaderSpec.groovy
@@ -21,6 +21,7 @@ package org.grails.datastore.gorm.jdbc.sqltail.cli
import org.grails.datastore.gorm.jdbc.sqltail.SqlTailRecord
import spock.lang.Specification
import spock.lang.TempDir
+import spock.lang.Unroll
class SqlTailFileReaderSpec extends Specification {
@@ -103,6 +104,40 @@ class SqlTailFileReaderSpec extends Specification {
records*.sql == ['select 10']
}
+ @Unroll
+ void "a file #what after the reader measured it is a restart, not a failure"() {
+ given: 'a reader that has read the file'
+ file.text = header('run-1') + line('r1', 'select 1') + '\n'
+ TruncatedWhenMeasured capture = new TruncatedWhenMeasured(file, replacement)
+ SqlTailFileReader reader = new SqlTailFileReader(capture, true)
+ reader.poll()
+
+ when: 'the application has appended to the file, and truncates it once the reader has measured that'
+ file << line('r1', 'select 2') + '\n'
+ capture.truncateWhenMeasured = true
+ List<SqlTailRecord> records = reader.poll()
+
+ then:
+ records.isEmpty()
+ reader.restarted
+ reader.position == 0
+ reader.run == null
+
+ when: 'the new run is written'
+ file.text = header('run-2') + line('r1', 'select 3') + '\n'
+ records = reader.poll()
+
+ then:
+ !reader.restarted
+ reader.run == 'run-2'
+ records*.sql == ['select 3']
+
+ where:
+ what | replacement
+ 'truncated' | '# sql-tail cap'
+ 'removed' | null
+ }
+
void "a new run that has grown past where the reader was is read from its start"() {
given:
file.text = header('run-1') + line('r1', 'select 1') + '\n' + line('r1', 'select 2') + '\n'
@@ -194,3 +229,33 @@ class SqlTailFileReaderSpec extends Specification {
request: 'GET /book/show/1', sql: sql).toLine()
}
}
+
+/**
+ * A capture file the application truncates, or removes, the moment the reader has measured it: the
+ * length the reader goes on to read is then longer than the file.
+ */
+class TruncatedWhenMeasured extends File {
+
+ boolean truncateWhenMeasured
+ private final String replacement
+
+ TruncatedWhenMeasured(File file, String replacement) {
+ super(file.path)
+ this.replacement = replacement
+ }
+
+ @Override
+ long length() {
+ long length = super.length()
+ if (truncateWhenMeasured) {
+ truncateWhenMeasured = false
+ if (replacement == null) {
+ delete()
+ }
+ else {
+ setText(replacement)
+ }
+ }
+ length
+ }
+}
diff --git a/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailCaptureSpec.groovy b/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailCaptureSpec.groovy
index ae021aa5d4..a24274fb66 100644
--- a/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailCaptureSpec.groovy
+++ b/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailCaptureSpec.groovy
@@ -205,6 +205,20 @@ class SqlTailCaptureSpec extends Specification {
!file.text.contains('example-secret')
}
+ void "a triple-quoted value is not recorded from a GoogleSQL database, whose strings hold quotes that way"() {
+ given:
+ File file = new File(tempDir, 'capture.log')
+ SqlTailCapture capture = open(file)
+ capture.syntax = SqlTailRedactor.Syntax.GOOGLE_SQL
+
+ when:
+ capture.record('DEFAULT', 'select """a"example-secret"b""" from t')
+
+ then:
+ records(file)*.sql == ['select ? from t']
+ !file.text.contains('example-secret')
+ }
+
void "the capture named by the settings is opened"() {
given:
File file = new File(tempDir, 'from-settings.log')
diff --git a/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactorSpec.groovy b/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactorSpec.groovy
index 27abc5166c..c0d0826e92 100644
--- a/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactorSpec.groovy
+++ b/grails-datamapping-core/src/test/groovy/org/grails/datastore/gorm/jdbc/sqltail/SqlTailRedactorSpec.groovy
@@ -21,6 +21,7 @@ package org.grails.datastore.gorm.jdbc.sqltail
import spock.lang.Specification
import spock.lang.Unroll
+import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.GOOGLE_SQL
import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.MYSQL
import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.STANDARD
import static org.grails.datastore.gorm.jdbc.sqltail.SqlTailRedactor.Syntax.TRANSACT_SQL
@@ -79,6 +80,24 @@ class SqlTailRedactorSpec extends Specification {
'select "it\'s ""example-secret""" from [t]' || 'select ? from [t]'
}
+ @Unroll
+ void "GoogleSQL: #sql"() {
+ expect:
+ redact(sql, GOOGLE_SQL) == redacted
+
+ where:
+ sql || redacted
+ 'select """a"example-secret"b""" from t' || 'select ? from t'
+ "select '''it's bob's example-secret''', 'x' from t" || 'select ?, ? from t'
+ 'select `account`.`id` from `account` where password = "example-secret"' || 'select `account`.`id` from `account` where password = ?'
+ "select 'it\\'s', \"say \\\"hi\\\"\", `we\\`ird` from t" || 'select ?, ?, `we\\`ird` from t'
+ "select r'\\d+', b'ab', rb'x', R\"\"\"a\"\"\" from t" || 'select r?, b?, rb?, R? from t'
+ "select '', 'it''s' from t" || 'select ?, ?? from t'
+ 'select 1 # key=abc123\nfrom t' || 'select ? from t'
+ 'select 1--key=abc123\nfrom t' || 'select ? from t'
+ "select 1 /* a /* b */ 'example-secret' */" || 'select ? ? */'
+ }
+
@Unroll
void "a statement whose values cannot be told apart from its SQL is withheld: #reason"() {
expect:
@@ -93,8 +112,12 @@ class SqlTailRedactorSpec extends Specification {
'a quote a backslash may or may not escape' | STANDARD | "select 'it\\'s', 'example-secret'"
'a comment that may or may not nest' | STANDARD | "select 1 /* a /* b */ 'example-secret' */"
'a quote a backslash escapes unless the server says not' | MYSQL | "select 'it\\'s'"
+ 'a triple-quoted string that never ends' | GOOGLE_SQL | "select '''example-secret'' from t"
+ 'a quoted identifier a backslash escapes in' | GOOGLE_SQL | 'select `example-secret\\` from t'
'double quotes before the dialect is known' | UNKNOWN | 'select "example-secret" from t'
'a hash before the dialect is known' | UNKNOWN | 'select 1 # example-secret'
+ 'a triple-quoted string before the dialect is known' | UNKNOWN | "select '''a'example-secret'b''' from t"
+ 'a doubled quote before the dialect is known' | UNKNOWN | "select 'it''s', 'example-secret' from t"
}
@Unroll
@@ -132,6 +155,24 @@ class SqlTailRedactorSpec extends Specification {
syntax << SqlTailRedactor.Syntax.values()
}
+ void "a value written into a GoogleSQL statement never survives redaction"() {
+ given:
+ List<String> statements = [
+ "select '''it's ${SECRET}''' from t",
+ "select '''it's bob's ${SECRET}''' from t",
+ "select \"\"\"a\"${SECRET}\"b\"\"\" from t",
+ "select \"${SECRET}\" from t",
+ "select 'it\\'s ${SECRET}' from t",
+ "select r'${SECRET}\\' from t",
+ "select 'a', '''${SECRET}",
+ "select 1 # ${SECRET}",
+ ]*.toString()
+
+ expect: 'in the syntax, and before it is known'
+ statements.every { !SqlTailRedactor.redact(it, GOOGLE_SQL).contains(SECRET) }
+ statements.every { !SqlTailRedactor.redact(it, UNKNOWN).contains(SECRET) }
+ }
+
void "nothing is nothing"() {
expect:
SqlTailRedactor.redact(null, STANDARD) == null
diff --git a/grails-doc/src/en/guide/conf/dataSource/sqlTail.adoc b/grails-doc/src/en/guide/conf/dataSource/sqlTail.adoc
index ba34be06b8..23a7b74727 100644
--- a/grails-doc/src/en/guide/conf/dataSource/sqlTail.adoc
+++ b/grails-doc/src/en/guide/conf/dataSource/sqlTail.adoc
@@ -84,11 +84,14 @@ literal or comment that never ends, is recorded as
Double-quoted text is a value on some databases and a name on others. MySQL and MariaDB read it as
a string, and so do SQL Server and Sybase when `QUOTED_IDENTIFIER` is off, which is Sybase's
default, so on those databases it is recorded as `?`, even when a statement written by hand uses it
-to quote a name. It is kept as a name only with the dialects of H2, PostgreSQL, CockroachDB,
-Oracle, DB2, HSQLDB and HANA, and of Derby with Hibernate 5, which read it only as a name. Any
-other dialect, such as SQLite's or Informix's, whose databases read double-quoted text as a string
-by default, is read in every syntax, so a statement that quotes a name in double quotes is withheld
-there.
+to quote a name. Cloud Spanner's GoogleSQL reads it as a string too, and a string there can be
+triple-quoted and then hold quotes of its own, which is how statements are read with Hibernate 7's
+Spanner dialect. Double-quoted text is kept as a name only with the dialects of H2, PostgreSQL,
+CockroachDB, Oracle, DB2, HSQLDB and HANA, and of Derby with Hibernate 5, which read it only as a
+name. Any other dialect, such as SQLite's or Informix's, whose databases read double-quoted text as
+a string by default, or a dialect that is not Hibernate's, such as the Spanner dialect with
+Hibernate 5, is read in every syntax, so a statement that quotes a name in double quotes, doubles a
+quote inside a string, or triple-quotes a string is withheld there.
Capture is off unless the application is started with it. An application started without the
flag carries no statement inspector, so it costs nothing in production or in development. The