You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by ggthename <gi...@git.apache.org> on 2017/06/23 04:18:58 UTC
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
GitHub user ggthename opened a pull request:
https://github.com/apache/nifi/pull/1937
NIFI-4105 support the specified Maximum value column and CSV Stream for Cassandra
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ggthename/nifi nifi-4105
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1937.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1937
----
commit aaabe8856d147368ad722732a0646a6a0104102d
Author: yoonwon.ko <le...@gmail.com>
Date: 2017-06-22T10:24:38Z
NIFI-4105 Adding classes based on QueryCassandra
commit b8c132d720e13835a4d03b39fe90e4e14f11f2b4
Author: yoonwon.ko <le...@gmail.com>
Date: 2017-06-23T01:50:44Z
NIFI-4105 the specified Maximum value column and CSV Stream for Cassandra
commit 3fb0234fb52b8ed9aba3b5a9ff1e77496b7a48f4
Author: yoonwon.ko <le...@gmail.com>
Date: 2017-06-23T01:59:12Z
NIFI-4105 support the specified Maximum value column and CSV Stream for Cassandra
commit 575c774155e339e5882e222152f32be45b4c0b95
Author: yoonwon.ko <le...@gmail.com>
Date: 2017-06-23T04:14:39Z
NIFI-4105 set state (min,max value of the specified column)
commit a87fd6c50e0bcd019d5e0f9774f163539ab96a5c
Author: yoonwon.ko <le...@gmail.com>
Date: 2017-06-23T04:17:50Z
NIFI-4105 add comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/1937
No worries @ggthename, I still see some issues regarding Keyspace property not evaluated with expression language:
````
Running org.apache.nifi.processors.cassandra.PutCassandraQLTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.189 sec - in org.apache.nifi.processors.cassandra.PutCassandraQLTest
Results :
Failed tests:
QueryCassandraTest.testProcessorELConfigJsonOutput:173 java.lang.IllegalStateException: Attempting to retrieve value of PropertyDescriptor[Keyspace] without first evaluating Expressions, even though the PropertyDescriptor indicates that the Expression Language is Supported. If you realize that this is the case and do not want this error to occur, it can be disabled by calling TestRunner.setValidateExpressionUsage(false)
QueryCassandraTest.testProcessorEmptyFlowFileAndExceptions:212 java.lang.IllegalStateException: Attempting to retrieve value of PropertyDescriptor[Keyspace] without first evaluating Expressions, even though the PropertyDescriptor indicates that the Expression Language is Supported. If you realize that this is the case and do not want this error to occur, it can be disabled by calling TestRunner.setValidateExpressionUsage(false)
QueryCassandraTest.testProcessorJsonOutput:136 java.lang.IllegalStateException: Attempting to retrieve value of PropertyDescriptor[Keyspace] without first evaluating Expressions, even though the PropertyDescriptor indicates that the Expression Language is Supported. If you realize that this is the case and do not want this error to occur, it can be disabled by calling TestRunner.setValidateExpressionUsage(false)
QueryCassandraTest.testProcessorJsonOutputWithQueryTimeout:197 java.lang.IllegalStateException: Attempting to retrieve value of PropertyDescriptor[Keyspace] without first evaluating Expressions, even though the PropertyDescriptor indicates that the Expression Language is Supported. If you realize that this is the case and do not want this error to occur, it can be disabled by calling TestRunner.setValidateExpressionUsage(false)
QueryCassandraTest.testProcessorNoInputFlowFileAndExceptions:97 java.lang.IllegalStateException: Attempting to retrieve value of PropertyDescriptor[Keyspace] without first evaluating Expressions, even though the PropertyDescriptor indicates that the Expression Language is Supported. If you realize that this is the case and do not want this error to occur, it can be disabled by calling TestRunner.setValidateExpressionUsage(false)
````
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124734304
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -75,19 +81,64 @@
@Tags({"cassandra", "cql", "select"})
@EventDriven
@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
-@CapabilityDescription("Execute provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. Query result "
- + "may be converted to Avro or JSON format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+@CapabilityDescription("Executes provided Cassandra Query Language (CQL) select query on a Cassandra to fetch all rows whose values"
+ + "in the specified Maximum Value column(s) are larger than the previously-seen maxima.Query result"
+ + "may be converted to Avro, JSON or CSV format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+ "scheduled to run on a timer, or cron expression, using the standard scheduling methods, or it can be triggered by an incoming FlowFile. "
+ "If it is triggered by an incoming FlowFile, then attributes of that FlowFile will be available when evaluating the "
+ "select query. FlowFile attribute 'executecql.row.count' indicates how many rows were selected.")
+@Stateful(scopes = Scope.CLUSTER, description = "After performing query, the maximum value of the specified column is stored, "
+ + "fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maximum"
+ + "State is stored across the cluster so that the next time this Processor can be run with min and max values")
@WritesAttributes({@WritesAttribute(attribute = "executecql.row.count", description = "The number of rows returned by the CQL query")})
public class QueryCassandra extends AbstractCassandraProcessor {
+ public static final String CSV_FORMAT = "CSV";
public static final String AVRO_FORMAT = "Avro";
public static final String JSON_FORMAT = "JSON";
+ public static final String CASSANDRA_WATERMARK_MIN_VALUE_ID = "CASSANDRA_WATERMARK_MIN_VALUE_ID";
+ public static final String CASSANDRA_WATERMARK_MAX_VALUE_ID = "CASSANDRA_WATERMARK_MAX_VALUE_ID";
+
public static final String RESULT_ROW_COUNT = "executecql.row.count";
+ public static final PropertyDescriptor INIT_WATERMARK = new PropertyDescriptor.Builder().name("Initial Watermark Value")
+ .description("Use it only once.")
+ .required(false)
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+ .build();
+
+ public static final PropertyDescriptor BACKOFF_PERIOD = new PropertyDescriptor.Builder()
+ .name("Backoff Period")
+ .description("Only records older than the backoff period will be eligible for pickup. This can be used in the ILM use case to define a retention period.")
+ .defaultValue("10 seconds")
+ .required(true)
+ .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+ .sensitive(false)
+ .build();
+
+ public static final PropertyDescriptor OVERLAP_TIME = new PropertyDescriptor.Builder()
+ .name("Overlap Period")
+ .description("Amount of time to overlap into the last load date to ensure long running transactions missed by previous load weren't missed. Recommended: >0s")
+ .required(true)
+ .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+ .defaultValue("0 seconds")
+ .build();
+
+ public static final PropertyDescriptor DATE_FIELD = new PropertyDescriptor.Builder()
--- End diff --
You're only accepting a date column, but that also could be an ID, no? Also, see my comment about potentially using multiple columns.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/1937
@ggthename, yes you can ignore this test failure, we are aware that this test intermittently fails. If you prefer you can do a ``mvn clean install -Pcontrib-check`` only in the Cassandra bundle directory for now.
The tests are now OK but the build still fails because of checkstyle issues (you need to run the ``contrib-check`` maven profile for that):
````
[WARNING] src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java[38] (imports) AvoidStarImport: Using the '.*' form of import should be avoided - org.apache.nifi.annotation.behavior.*.
[WARNING] src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java[62] (imports) AvoidStarImport: Using the '.*' form of import should be avoided - java.util.*.
[WARNING] src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java[442:13] (blocks) RightCurly: '}' should be on the same line.
[WARNING] src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java[451:9] (blocks) RightCurly: '}' should be on the same line.
[WARNING] src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java[483:13] (blocks) RightCurly: '}' should be on the same line.
[WARNING] src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java[721:9] (blocks) RightCurly: '}' should be on the same line.
````
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124735002
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -485,6 +668,68 @@ public static long convertToJsonStream(final ResultSet rs, final OutputStream ou
}
}
+ /**
+ * Converts a result set into an CSV record and writes it to the given stream using the specified character set.
+ *
+ * @param rs The result set to convert
+ * @param outStream The stream to which the CSV record will be written
+ * @param timeout The max number of timeUnits to wait for a result set fetch to complete
+ * @param timeUnit The unit of time (SECONDS, e.g.) associated with the timeout amount
+ * @return The number of rows from the result set written to the stream
+ * @throws IOException If the CSV record cannot be written
+ * @throws InterruptedException If a result set fetch is interrupted
+ * @throws TimeoutException If a result set fetch has taken longer than the specified timeout
+ * @throws ExecutionException If any error occurs during the result set fetch
+ */
+ public static long convertToCsvStream(final ResultSet rs, final OutputStream outStream, Charset charset,
--- End diff --
Should we write a header line with the name of the columns? Or add a boolean in the processor properties to define if the header should be included or not in the generated flow file?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124733832
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -75,19 +81,64 @@
@Tags({"cassandra", "cql", "select"})
@EventDriven
@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
-@CapabilityDescription("Execute provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. Query result "
- + "may be converted to Avro or JSON format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+@CapabilityDescription("Executes provided Cassandra Query Language (CQL) select query on a Cassandra to fetch all rows whose values"
--- End diff --
A bit confused by the wording, maybe something like:
> Executes provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. In addition, incremental fetching can be achieved by setting Maximum-Value Columns, which causes the processor to track the columns' maximum values, thus only fetching rows whose columns' values exceed the observed maximums. This processor is intended to be run on the Primary Node only.
Also be careful with the white spaces.
In my proposition I'm talking about multiple maximum column values, but it seems you only accept one column. You can have a look at GenerateTableFetch processor where a user can define multiple columns to track new rows.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124944247
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -213,13 +308,20 @@ public void onTrigger(final ProcessContext context, final ProcessSession session
return;
}
}
-
- final ComponentLog logger = getLogger();
final String selectQuery = context.getProperty(CQL_SELECT_QUERY).evaluateAttributeExpressions(fileToProcess).getValue();
final long queryTimeout = context.getProperty(QUERY_TIMEOUT).evaluateAttributeExpressions(fileToProcess).asTimePeriod(TimeUnit.MILLISECONDS);
final String outputFormat = context.getProperty(OUTPUT_FORMAT).getValue();
final Charset charset = Charset.forName(context.getProperty(CHARSET).evaluateAttributeExpressions(fileToProcess).getValue());
final StopWatch stopWatch = new StopWatch(true);
+ final String waterMarkDateField = context.getProperty(DATE_FIELD).getValue();
+ final String tableName = context.getProperty(TABLE_NAME).getValue();
+ final String keySpace = context.getProperty(KEYSPACE).evaluateAttributeExpressions(fileToProcess).getValue();
+
+ if ( StringUtils.isEmpty(selectQuery) && StringUtils.isEmpty(tableName) ) {
--- End diff --
Yes. you're right.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124962998
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/QueryCassandraTest.java ---
@@ -361,6 +362,14 @@ public void testConvertToJSONStream() throws Exception {
assertEquals(2, numberOfRows);
}
+ @Test
+ public void testConvertToCSVStream() throws Exception {
+ ResultSet rs = CassandraQueryTestUtil.createMockResultSet();
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ long numberOfRows = QueryCassandra.convertToCsvStream(rs, baos, StandardCharsets.UTF_8, 0, null);
+ assertEquals(2, numberOfRows);
--- End diff --
I'll add a test case which is called testProcessorCsvOutput.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on the issue:
https://github.com/apache/nifi/pull/1937
@pvillard31 I also solved checkstyle issues after doing a mvn clean install -Pcontrib-check.
Do you have still issues? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124734510
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -213,13 +308,20 @@ public void onTrigger(final ProcessContext context, final ProcessSession session
return;
}
}
-
- final ComponentLog logger = getLogger();
final String selectQuery = context.getProperty(CQL_SELECT_QUERY).evaluateAttributeExpressions(fileToProcess).getValue();
final long queryTimeout = context.getProperty(QUERY_TIMEOUT).evaluateAttributeExpressions(fileToProcess).asTimePeriod(TimeUnit.MILLISECONDS);
final String outputFormat = context.getProperty(OUTPUT_FORMAT).getValue();
final Charset charset = Charset.forName(context.getProperty(CHARSET).evaluateAttributeExpressions(fileToProcess).getValue());
final StopWatch stopWatch = new StopWatch(true);
+ final String waterMarkDateField = context.getProperty(DATE_FIELD).getValue();
+ final String tableName = context.getProperty(TABLE_NAME).getValue();
+ final String keySpace = context.getProperty(KEYSPACE).evaluateAttributeExpressions(fileToProcess).getValue();
+
+ if ( StringUtils.isEmpty(selectQuery) && StringUtils.isEmpty(tableName) ) {
--- End diff --
I think this should be evaluated in customValidate() method instead of onTrigger().
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124735339
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/QueryCassandraTest.java ---
@@ -361,6 +362,14 @@ public void testConvertToJSONStream() throws Exception {
assertEquals(2, numberOfRows);
}
+ @Test
+ public void testConvertToCSVStream() throws Exception {
--- End diff --
Could you also add unit tests for the incremental fetch feature?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124841474
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -75,19 +81,64 @@
@Tags({"cassandra", "cql", "select"})
@EventDriven
@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
-@CapabilityDescription("Execute provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. Query result "
- + "may be converted to Avro or JSON format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+@CapabilityDescription("Executes provided Cassandra Query Language (CQL) select query on a Cassandra to fetch all rows whose values"
--- End diff --
ORDER BY and ROWNUM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on the issue:
https://github.com/apache/nifi/pull/1937
Thank you for your opinion. There were lots of conflicts.. I finished rebase opertaion.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124735162
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/QueryCassandraTest.java ---
@@ -361,6 +362,14 @@ public void testConvertToJSONStream() throws Exception {
assertEquals(2, numberOfRows);
}
+ @Test
+ public void testConvertToCSVStream() throws Exception {
+ ResultSet rs = CassandraQueryTestUtil.createMockResultSet();
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ long numberOfRows = QueryCassandra.convertToCsvStream(rs, baos, StandardCharsets.UTF_8, 0, null);
+ assertEquals(2, numberOfRows);
--- End diff --
Could you also check the output format and not only the number of rows?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on the issue:
https://github.com/apache/nifi/pull/1937
@pvillard31 I have one question.
There were some problems on my side(maven 3.5.0)
What is the meaning of failure? How can I solve this situation. Thanks!
`[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.
18:test (default-test) on project nifi-write-ahead-log: There are test failures.
[ERROR]
[ERROR] Please refer to D:\github\nifi\nifi-commons\nifi-write-ahead-log\target\
surefire-reports for the individual test results.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal o
rg.apache.maven.plugins:maven-surefire-plugin:2.18:test (default-test) on projec
t nifi-write-ahead-log: There are test failures.
Please refer to D:\github\nifi\nifi-commons\nifi-write-ahead-log\target\surefire
-reports for the individual test results.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor
.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor
.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor
.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje
ct(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje
ct(LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThre
adedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(Lifecycl
eStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Laun
cher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav
a:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La
uncher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:
356)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures
.
`
and It is the TestMinimalLockingWriteAheadLog.txt
`
-------------------------------------------------------------------------------
Test set: org.wali.TestMinimalLockingWriteAheadLog
-------------------------------------------------------------------------------
Tests run: 12, Failures: 0, Errors: 1, Skipped: 2, Time elapsed: 7.641 sec <<< FAILURE! - in org.wali.TestMinimalLockingWriteAheadLog
testRecoverFileThatHasTrailingNULBytesAndTruncation(org.wali.TestMinimalLockingWriteAheadLog) Time elapsed: 0.03 sec <<< ERROR!
java.nio.channels.OverlappingFileLockException: null
at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187)
at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:108)
at org.wali.TestMinimalLockingWriteAheadLog.testRecoverFileThatHasTrailingNULBytesAndTruncation(TestMinimalLockingWriteAheadLog.java:472)
`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by mattyb149 <gi...@git.apache.org>.
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/1937
This PR seems to have some conflicts, can you rebase your commits against the latest master? Please and thanks!
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124765825
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -75,19 +81,64 @@
@Tags({"cassandra", "cql", "select"})
@EventDriven
@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
-@CapabilityDescription("Execute provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. Query result "
- + "may be converted to Avro or JSON format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+@CapabilityDescription("Executes provided Cassandra Query Language (CQL) select query on a Cassandra to fetch all rows whose values"
+ + "in the specified Maximum Value column(s) are larger than the previously-seen maxima.Query result"
+ + "may be converted to Avro, JSON or CSV format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+ "scheduled to run on a timer, or cron expression, using the standard scheduling methods, or it can be triggered by an incoming FlowFile. "
+ "If it is triggered by an incoming FlowFile, then attributes of that FlowFile will be available when evaluating the "
+ "select query. FlowFile attribute 'executecql.row.count' indicates how many rows were selected.")
+@Stateful(scopes = Scope.CLUSTER, description = "After performing query, the maximum value of the specified column is stored, "
+ + "fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maximum"
+ + "State is stored across the cluster so that the next time this Processor can be run with min and max values")
@WritesAttributes({@WritesAttribute(attribute = "executecql.row.count", description = "The number of rows returned by the CQL query")})
public class QueryCassandra extends AbstractCassandraProcessor {
+ public static final String CSV_FORMAT = "CSV";
public static final String AVRO_FORMAT = "Avro";
public static final String JSON_FORMAT = "JSON";
+ public static final String CASSANDRA_WATERMARK_MIN_VALUE_ID = "CASSANDRA_WATERMARK_MIN_VALUE_ID";
+ public static final String CASSANDRA_WATERMARK_MAX_VALUE_ID = "CASSANDRA_WATERMARK_MAX_VALUE_ID";
+
public static final String RESULT_ROW_COUNT = "executecql.row.count";
+ public static final PropertyDescriptor INIT_WATERMARK = new PropertyDescriptor.Builder().name("Initial Watermark Value")
+ .description("Use it only once.")
--- End diff --
I'm not familiar with english. But I'll try again!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124961912
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -485,6 +668,68 @@ public static long convertToJsonStream(final ResultSet rs, final OutputStream ou
}
}
+ /**
+ * Converts a result set into an CSV record and writes it to the given stream using the specified character set.
+ *
+ * @param rs The result set to convert
+ * @param outStream The stream to which the CSV record will be written
+ * @param timeout The max number of timeUnits to wait for a result set fetch to complete
+ * @param timeUnit The unit of time (SECONDS, e.g.) associated with the timeout amount
+ * @return The number of rows from the result set written to the stream
+ * @throws IOException If the CSV record cannot be written
+ * @throws InterruptedException If a result set fetch is interrupted
+ * @throws TimeoutException If a result set fetch has taken longer than the specified timeout
+ * @throws ExecutionException If any error occurs during the result set fetch
+ */
+ public static long convertToCsvStream(final ResultSet rs, final OutputStream outStream, Charset charset,
--- End diff --
I think it should be optional. Because other processors have to connect with this processor using flowfile.
So I'll adding a boolean value.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on the issue:
https://github.com/apache/nifi/pull/1937
@pvillard31 I fixed those tests. But I found the failures on other components. Is it ok?
`[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.
18:test (default-test) on project nifi-write-ahead-log: There are test failures.
[ERROR]
[ERROR] Please refer to D:\github\nifi\nifi-commons\nifi-write-ahead-log\target\
surefire-reports for the individual test results.
-------------------------------------------------------------------------------
Test set: org.wali.TestMinimalLockingWriteAheadLog
-------------------------------------------------------------------------------
Tests run: 12, Failures: 0, Errors: 1, Skipped: 2, Time elapsed: 7.56 sec <<< FAILURE! - in org.wali.TestMinimalLockingWriteAheadLog
testRecoverFileThatHasTrailingNULBytesAndTruncation(org.wali.TestMinimalLockingWriteAheadLog) Time elapsed: 0.032 sec <<< ERROR!
java.nio.channels.OverlappingFileLockException: null
at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187)
at org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:108)
at org.wali.TestMinimalLockingWriteAheadLog.testRecoverFileThatHasTrailingNULBytesAndTruncation(TestMinimalLockingWriteAheadLog.java:472)
`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124964237
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -75,19 +81,64 @@
@Tags({"cassandra", "cql", "select"})
@EventDriven
@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
-@CapabilityDescription("Execute provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. Query result "
- + "may be converted to Avro or JSON format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+@CapabilityDescription("Executes provided Cassandra Query Language (CQL) select query on a Cassandra to fetch all rows whose values"
+ + "in the specified Maximum Value column(s) are larger than the previously-seen maxima.Query result"
+ + "may be converted to Avro, JSON or CSV format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+ "scheduled to run on a timer, or cron expression, using the standard scheduling methods, or it can be triggered by an incoming FlowFile. "
+ "If it is triggered by an incoming FlowFile, then attributes of that FlowFile will be available when evaluating the "
+ "select query. FlowFile attribute 'executecql.row.count' indicates how many rows were selected.")
+@Stateful(scopes = Scope.CLUSTER, description = "After performing query, the maximum value of the specified column is stored, "
+ + "fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maximum"
+ + "State is stored across the cluster so that the next time this Processor can be run with min and max values")
@WritesAttributes({@WritesAttribute(attribute = "executecql.row.count", description = "The number of rows returned by the CQL query")})
public class QueryCassandra extends AbstractCassandraProcessor {
+ public static final String CSV_FORMAT = "CSV";
public static final String AVRO_FORMAT = "Avro";
public static final String JSON_FORMAT = "JSON";
+ public static final String CASSANDRA_WATERMARK_MIN_VALUE_ID = "CASSANDRA_WATERMARK_MIN_VALUE_ID";
+ public static final String CASSANDRA_WATERMARK_MAX_VALUE_ID = "CASSANDRA_WATERMARK_MAX_VALUE_ID";
+
public static final String RESULT_ROW_COUNT = "executecql.row.count";
+ public static final PropertyDescriptor INIT_WATERMARK = new PropertyDescriptor.Builder().name("Initial Watermark Value")
+ .description("Use it only once.")
+ .required(false)
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+ .build();
+
+ public static final PropertyDescriptor BACKOFF_PERIOD = new PropertyDescriptor.Builder()
+ .name("Backoff Period")
+ .description("Only records older than the backoff period will be eligible for pickup. This can be used in the ILM use case to define a retention period.")
+ .defaultValue("10 seconds")
+ .required(true)
+ .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+ .sensitive(false)
+ .build();
+
+ public static final PropertyDescriptor OVERLAP_TIME = new PropertyDescriptor.Builder()
+ .name("Overlap Period")
+ .description("Amount of time to overlap into the last load date to ensure long running transactions missed by previous load weren't missed. Recommended: >0s")
+ .required(true)
+ .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
+ .defaultValue("0 seconds")
+ .build();
+
+ public static final PropertyDescriptor DATE_FIELD = new PropertyDescriptor.Builder()
--- End diff --
I think it's a good idea, but a little bit complex.
because we usually want to concentrate the time of record when we use the incremental fetch feature. ( To avoid omissions )
So there are the concepts of BACKOFF_PERIOD and OVERLAP_TIME.
It is difficult to apply the other type column.
In my opinion, a date column is most appropriate for the incremental fetch feature.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/1937
Hi @ggthename, thanks for your contribution. Can you check that
``mvn clean install -Pcontrib-check``
is succeeding on your side? Travis builds are complaining about test failures and I've the same on my side. Once you have a succeeding build, can you rebase your PR and squash your commits.
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124964375
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/test/java/org/apache/nifi/processors/cassandra/QueryCassandraTest.java ---
@@ -361,6 +362,14 @@ public void testConvertToJSONStream() throws Exception {
assertEquals(2, numberOfRows);
}
+ @Test
+ public void testConvertToCSVStream() throws Exception {
--- End diff --
Sure! I forgot to push.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on the issue:
https://github.com/apache/nifi/pull/1937
@pvillard31
Really? I'm sorry. I'll check again. Thank you.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124734093
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -75,19 +81,64 @@
@Tags({"cassandra", "cql", "select"})
@EventDriven
@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
-@CapabilityDescription("Execute provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. Query result "
- + "may be converted to Avro or JSON format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+@CapabilityDescription("Executes provided Cassandra Query Language (CQL) select query on a Cassandra to fetch all rows whose values"
+ + "in the specified Maximum Value column(s) are larger than the previously-seen maxima.Query result"
+ + "may be converted to Avro, JSON or CSV format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+ "scheduled to run on a timer, or cron expression, using the standard scheduling methods, or it can be triggered by an incoming FlowFile. "
+ "If it is triggered by an incoming FlowFile, then attributes of that FlowFile will be available when evaluating the "
+ "select query. FlowFile attribute 'executecql.row.count' indicates how many rows were selected.")
+@Stateful(scopes = Scope.CLUSTER, description = "After performing query, the maximum value of the specified column is stored, "
+ + "fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maximum"
+ + "State is stored across the cluster so that the next time this Processor can be run with min and max values")
@WritesAttributes({@WritesAttribute(attribute = "executecql.row.count", description = "The number of rows returned by the CQL query")})
public class QueryCassandra extends AbstractCassandraProcessor {
+ public static final String CSV_FORMAT = "CSV";
public static final String AVRO_FORMAT = "Avro";
public static final String JSON_FORMAT = "JSON";
+ public static final String CASSANDRA_WATERMARK_MIN_VALUE_ID = "CASSANDRA_WATERMARK_MIN_VALUE_ID";
+ public static final String CASSANDRA_WATERMARK_MAX_VALUE_ID = "CASSANDRA_WATERMARK_MAX_VALUE_ID";
+
public static final String RESULT_ROW_COUNT = "executecql.row.count";
+ public static final PropertyDescriptor INIT_WATERMARK = new PropertyDescriptor.Builder().name("Initial Watermark Value")
+ .description("Use it only once.")
--- End diff --
Can you put .name() on a new line like the other properties. Also, can you use .displayName() for what will be displayed at the UI and .name() for a more "computer-friendly" name? That's not mandatory but it's a convention we're trying to adopt. Same for other new properties.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1937: NIFI-4105 support the specified Maximum value column and C...
Posted by mattyb149 <gi...@git.apache.org>.
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/1937
Sorry we lost track of this, I can review again but first can you rebase your commits against the latest master? Please and thanks!
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by ggthename <gi...@git.apache.org>.
Github user ggthename commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124765427
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -75,19 +81,64 @@
@Tags({"cassandra", "cql", "select"})
@EventDriven
@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
-@CapabilityDescription("Execute provided Cassandra Query Language (CQL) select query on a Cassandra 1.x, 2.x, or 3.0.x cluster. Query result "
- + "may be converted to Avro or JSON format. Streaming is used so arbitrarily large result sets are supported. This processor can be "
+@CapabilityDescription("Executes provided Cassandra Query Language (CQL) select query on a Cassandra to fetch all rows whose values"
--- End diff --
Thank you for your wording! I really appreciate it.
I think it is a little different from GenerateTableFetch. Because GenerateTableFetch use various kinds of DatabaseAdapter for RDB. They usually use "ORDER BY" keyword.
But Cassandra does not have this feature. And It has specialized structure based on Column Family, not Row
So I built this processor with range query.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1937: NIFI-4105 support the specified Maximum value colum...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1937#discussion_r124734623
--- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-processors/src/main/java/org/apache/nifi/processors/cassandra/QueryCassandra.java ---
@@ -310,9 +421,81 @@ public void process(final OutputStream out) throws IOException {
session.remove(fileToProcess);
context.yield();
}
+ } catch (final IOException e) {
+ logger.error("Unable to set state{}",
--- End diff --
Could you add the exception object ``e`` in the log?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---