Re: [I] Job graph fails to display in UI [arrow-ballista]

["simicd (via GitHub)" <gi...@apache.org>]

simicd commented on issue #627:
URL: https://github.com/apache/arrow-ballista/issues/627#issuecomment-1911128842

   Hi all, while working on #957 I faced the same issue, here's how to reproduce it:
   
   1. In the API handler, `get_job_svg_graph` function replace `.map_err(|_| warp::reject())` with `.map_err(|err|  { info!("Issue: {:?}", err); warp::reject()})`
   https://github.com/apache/arrow-ballista/blob/9903ab27f121f16717a70de7a30f643c4c45dd34/ballista/scheduler/src/api/handlers.rs#L343-L348
   2. Launch scheduler and executor
   3. Submit a query, e.g. the `standalone-sql.rs` but with a call to remote:
       ```rust
       
       use ballista::prelude::{BallistaConfig, BallistaContext, Result};
       use ballista_examples::test_util;
       use datafusion::execution::options::ParquetReadOptions;
       
       #[tokio::main]
       async fn main() -> Result<()> {
           let config = BallistaConfig::builder()
               .set("ballista.shuffle.partitions", "1")
               .build()?;
       
           let ctx = BallistaContext::remote("localhost", 50050, &config).await?;
       
           let testdata = test_util::examples_test_data();
       
           // register parquet file with the execution context
           ctx.register_parquet(
               "test",
               &format!("{testdata}/alltypes_plain.parquet"),
               ParquetReadOptions::default(),
           )
           .await?;
       
           let df = ctx.sql("select count(1) from test").await?;
       
           df.show().await?;
           Ok(())
       }
       ```
   4. Wait until query is executed
   5. Look up the job ID, e.g. GET request to `http://localhost:50050/api/jobs`
   6. Call `http://localhost:50050/api/job/{job_id}/dot_svg` with the retrieved job_id (e.g. `http://localhost:50050/api/job/bvppZ4r/dot_svg`)
   7. Go to the terminal of the scheduler and monitor the logs - I got the following error: 
       ```
       2024-01-25T22:28:24.800607Z  INFO tokio-runtime-worker ThreadId(19) ballista_scheduler::api::handlers: Issue: Error { kind: NotFound, message: "program not found" }
       ```
   
   Hope that helps to narrow it down!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org