[jira] [Updated] (BEAM-11027) ZetaSQL Nexmark run takes too long because of excessive logging

["Kenneth Knowles (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-11027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-11027:
-----------------------------------
    Resolution: Fixed
        Status: Resolved  (was: Resolved)

Hello! Due to a bug in our Jira configuration, this issue had status:Resolved but resolution:Unresolved.

I am bulk editing these issues to have resolution:Fixed

If a different resolution is appropriate, please change it. To do this, click the "Resolve" button (you can do this even for closed issues) and set the Resolution field to the right value.

> ZetaSQL Nexmark run takes too long because of excessive logging
> ---------------------------------------------------------------
>
>                 Key: BEAM-11027
>                 URL: https://issues.apache.org/jira/browse/BEAM-11027
>             Project: Beam
>          Issue Type: Test
>          Components: dsl-sql-zetasql, testing-nexmark
>            Reporter: Ismaël Mejía
>            Assignee: Ismaël Mejía
>            Priority: P2
>             Fix For: 2.26.0
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> The execution of the Nexmark ZetaSQL tests takes too long and produces a gigantic log file ~ 1.5GB full of debug information of netty/grpc when run on the Spark runner
> {quote}{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND HEADERS: streamId=39861 headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0 endStream=true}}
>  {{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND WINDOW_UPDATE: streamId=0 windowSizeIncrement=151}}
>  {{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] OUTBOUND HEADERS: streamId=39867 headers=GrpcHttp2OutboundHeaders[:authority: 0.0.0.0:0, :path: /zetasql.local_service.ZetaSqlLocalService/Evaluate, :method: POST, :scheme: http, content-type: application/grpc, te: trailers, user-agent: grpc-java-netty/1.18.0, grpc-accept-encoding: gzip] streamDependency=0 weight=16 exclusive=false padding=0 endStream=false}}
> {quote}
> The full run in the CI takes 2h30 because of this
>  [https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Spark_PR/52/]
> This can be reproduced by doing:
>   
>  ./gradlew :sdks:java:testing:nexmark:run -Pnexmark.runner=":runners:spark" -Pnexmark.args=" --runner=SparkRunner --streaming=false --suite=SMOKE --queryLanguage=zetaSql --manageResources=false --monitorJobs=true --enforceEncodability=true --enforceImmutability=true"



--
This message was sent by Atlassian Jira
(v8.3.4#803005)