You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 14:27:20 UTC
[GitHub] [beam] damccorm opened a new issue, #19927: What is corresponding data type to set in UDF parameter to match RecordType
damccorm opened a new issue, #19927:
URL: https://github.com/apache/beam/issues/19927
Hello Beam community,
I want to have an UDF to take a record as input. Per error info as shown below, it indicates that the input is *RecordType*, but ** what should I set in the UDF parameter so that Beam would not complain about the type compatibility? Below is the rull error trace. Thank you very much!
\==\==\==\==error trace\==\==\==\==\===
Exception in thread "main" org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable to parse query select fooudf(pv.header) from kafka.tracking.PageViewEvent as pvException in thread "main" org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable to parse query select fooudf(pv.header) from kafka.tracking.PageViewEvent as pv at org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:165) at org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.parseQuery(BeamSqlEnv.java:103) at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:124) at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:82) at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:539) at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:473) at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44) at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:169) at com.linkedin.samza.sql.engine.BeamSqlEntry.prepar
ePipeline(BeamSqlEntry.java:52) at com.linkedin.samza.sql.engine.BeamSqlEntry.exec(BeamSqlEntry.java:41) at com.linkedin.samza.sql.engine.BeamSqlUI.main(BeamSqlUI.java:33)Caused by: org.apache.calcite.tools.ValidationException: org.apache.calcite.runtime.CalciteContextException: From line 1, column 8 to line 1, column 24: No match found for function signature fooudf(<RecordType(INTEGER memberId, VARCHAR viewerUrn, VARCHAR applicationViewerUrn, VARCHAR csUserUrn, BIGINT time, VARCHAR server, VARCHAR service, VARCHAR environment, VARBINARY guid, VARBINARY treeId, INTEGER requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR instance, VARCHAR appName, VARCHAR testId, VARCHAR testSegmentId, RecordType(BIGINT time, VARCHAR server, VARCHAR instance, VARCHAR appName, VARBINARY messageId, INTEGER auditVersion, VARCHAR fabricUrn, VARCHAR clusterConnectionString) auditHeader, RecordType(VARCHAR pageUrn, VARBINARY trackingId) pageInstance, RecordType(VARCHAR applicationUrn, VARCHAR versi
on, VARBINARY trackingId) clientApplicationInstance, VARCHAR originSource, VARCHAR sessionUrn, RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId, VARCHAR rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context, DOUBLE scaleFactor) traceData)\>) at org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:190) at org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:136) ... 10 moreCaused by: org.apache.calcite.runtime.CalciteContextException: From line 1, column 8 to line 1, column 24: No match found for function signature fooudf(<RecordType(INTEGER memberId, VARCHAR viewerUrn, VARCHAR applicationViewerUrn, VARCHAR csUserUrn, BIGINT time, VARCHAR server, VARCHAR service, VARCHAR environment, VARBINARY guid, VARBINARY treeId, INTEGER requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR instance, VARCHAR appName, VARCHAR testId, VARCHAR testSegmentId, RecordType(BIGINT time, VARCHAR server, VARCHAR
instance, VARCHAR appName, VARBINARY messageId, INTEGER auditVersion, VARCHAR fabricUrn, VARCHAR clusterConnectionString) auditHeader, RecordType(VARCHAR pageUrn, VARBINARY trackingId) pageInstance, RecordType(VARCHAR applicationUrn, VARCHAR version, VARBINARY trackingId) clientApplicationInstance, VARCHAR originSource, VARCHAR sessionUrn, RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId, VARCHAR rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context, DOUBLE scaleFactor) traceData)\>) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil
.java:787) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:772) at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4825) at org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1739) at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:270) at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:215) at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5584) at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5571) at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138) at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1657) at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1642) at org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:462) at org.apache.calci
te.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4089) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3352) at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:994) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:954) at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:929) at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:633) at org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:188) ... 11 moreCaused by: org.apache.calcite.sql.validate.SqlValidatorException: No match found for function signature fooudf(<R
ecordType(INTEGER memberId, VARCHAR viewerUrn, VARCHAR applicationViewerUrn, VARCHAR csUserUrn, BIGINT time, VARCHAR server, VARCHAR service, VARCHAR environment, VARBINARY guid, VARBINARY treeId, INTEGER requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR instance, VARCHAR appName, VARCHAR testId, VARCHAR testSegmentId, RecordType(BIGINT time, VARCHAR server, VARCHAR instance, VARCHAR appName, VARBINARY messageId, INTEGER auditVersion, VARCHAR fabricUrn, VARCHAR clusterConnectionString) auditHeader, RecordType(VARCHAR pageUrn, VARBINARY trackingId) pageInstance, RecordType(VARCHAR applicationUrn, VARCHAR version, VARBINARY trackingId) clientApplicationInstance, VARCHAR originSource, VARCHAR sessionUrn, RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId, VARCHAR rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context, DOUBLE scaleFactor) traceData)\>) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstru
ctorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) ... 33 more
Imported from Jira [BEAM-8316](https://issues.apache.org/jira/browse/BEAM-8316). Original Jira may contain additional context.
Reported by: yangzhang.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] gabrywu commented on issue #19927: What is corresponding data type to set in UDF parameter to match RecordType
Posted by "gabrywu (via GitHub)" <gi...@apache.org>.
gabrywu commented on issue #19927:
URL: https://github.com/apache/beam/issues/19927#issuecomment-1631970421
any updates here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org