You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/10/31 14:44:00 UTC
[jira] [Work logged] (BEAM-6768) Support for data-dependent
destination for BQ streaming inserts in Python
[ https://issues.apache.org/jira/browse/BEAM-6768?focusedWorklogId=672459&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-672459 ]
ASF GitHub Bot logged work on BEAM-6768:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Oct/21 14:43
Start Date: 31/Oct/21 14:43
Worklog Time Spent: 10m
Work Description: ee07dazn commented on pull request #7677:
URL: https://github.com/apache/beam/pull/7677#issuecomment-955713230
@pabloem : Hi, looking for a way to automatically send back records to DLQ table, i came across [ticket](https://issues.apache.org/jira/browse/BEAM-4265), which references this PR... i believe what i get out from this is FAILED_ROWS as a list of tuples (destination, rows), which i need to process however i want to write to DLQ table as a separate transform - please correct me if i have understood anything wrong so far? Assuming thats correct, I believe we need the error message and/or stacktrace as part of that tuple such that its easier to look into and debug ...I do know that this currently goes into worker logs and i can look at the logs but it would be much easier to send the error message as part of the tuple....i believe the Java version of Pub/Sub--> BQ does this and has a set output BQ table schema that has payload(str), payload(bytes), errorMessage, stacktrace, timestamp...which all helps in debugging...let me know what you think ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 672459)
Time Spent: 1.5h (was: 1h 20m)
> Support for data-dependent destination for BQ streaming inserts in Python
> -------------------------------------------------------------------------
>
> Key: BEAM-6768
> URL: https://issues.apache.org/jira/browse/BEAM-6768
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Pablo Estrada
> Assignee: Pablo Estrada
> Priority: P2
> Fix For: 2.13.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)