You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "FelixYBW (via GitHub)" <gi...@apache.org> on 2024/07/20 06:53:36 UTC

[I] [VL] sanityCheckLoacked Error [incubator-gluten]

FelixYBW opened a new issue, #6528:
URL: https://github.com/apache/incubator-gluten/issues/6528

   ### Backend
   
   VL (Velox)
   
   ### Bug description
   
   ```24/07/20 06:46:25 ERROR [Executor task launch worker for task 980.0 in stage 0.0 (TID 980)] executor.Executor: Exception in task 980.0 in stage 0.0 (TID 980)
   org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
   Error Source: RUNTIME
   Error Code: INVALID_STATE
   Reason: Bad memory usage track state: Memory Pool[default_leaf LEAF root[root] parent[root] MALLOC track-usage thread-safe]<unlimited max capacity capacity 8.00MB used 12582912.00TB available 4194304.00TB reservation [used 12582912.00TB, reserved 0B, min 0B] counters [allocs 1, frees 0, reserves 0, releases 0, collisions 0])>
   Retriable: False
   Function: sanityCheckLocked
   File: /home/binweiyang/gluten/ep/build-velox/build/velox_ep/./velox/common/memory/MemoryPool.h
   Line: 919
   Stack trace:
   # 0  _ZN8facebook5velox7process10StackTraceC1Ei
   # 1  _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
   # 2  _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEEvRKNS1_18VeloxCheckFailArgsET0_
   # 3  _ZN8facebook5velox6memory14MemoryPoolImpl17reserveThreadSafeEmb
   # 4  _ZN8facebook5velox6memory14MemoryPoolImpl7reserveEmb
   # 5  _ZN8facebook5velox6memory14MemoryPoolImpl8allocateEl
   # 6  _ZN8facebook5velox13AlignedBuffer8allocateIhEEN5boost13intrusive_ptrINS0_6BufferEEEmPNS0_6memory10MemoryPoolERKSt8optionalIT_E
   # 7  _ZN6gluten27VeloxColumnarToRowConverter13refreshStatesESt10shared_ptrIN8facebook5velox9RowVectorEEll
   # 8  _ZN6gluten27VeloxColumnarToRowConverter7convertESt10shared_ptrINS_13ColumnarBatchEEll
   # 9  Java_org_apache_gluten_vectorized_NativeColumnarToRowJniWrapper_nativeColumnarToRowConvert
   # 10 0x00007ffbe71bf427
   
   	at org.apache.gluten.vectorized.NativeColumnarToRowJniWrapper.nativeColumnarToRowConvert(Native Method)
   	at org.apache.gluten.execution.VeloxColumnarToRowExec$$anon$1.next(VeloxColumnarToRowExec.scala:151)
   	at org.apache.gluten.execution.VeloxColumnarToRowExec$$anon$1.next(VeloxColumnarToRowExec.scala:117)```
   
   @zhztheplayer any insight?
   
   ### Spark version
   
   None
   
   ### Spark configurations
   
   _No response_
   
   ### System information
   
   _No response_
   
   ### Relevant logs
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL] sanityCheckLoacked Error [incubator-gluten]

Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #6528:
URL: https://github.com/apache/incubator-gluten/issues/6528#issuecomment-2241198039

   Looks the issue is caused by pr5952.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL] sanityCheckLoacked Error [incubator-gluten]

Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #6528:
URL: https://github.com/apache/incubator-gluten/issues/6528#issuecomment-2240965989

   query plan:
   ```
   Execute InsertIntoHadoopFsRelationCommand (15)
   +- FakeRowAdaptor (14)
      +- Project (13)
         +- VeloxColumnarToRowExec (12)
            +- ^ FilterExecTransformer (10)
               +- ^ GenerateExecTransformer (9)
                  +- ^ ProjectExecTransformer (8)
                     +- ^ FilterExecTransformer (7)
                        +- ^ GenerateExecTransformer (6)
                           +- ^ FilterExecTransformer (5)
                              +- ^ InputIteratorTransformer (4)
                                 +- RowToVeloxColumnar (2)
                                    +- Scan hive data.feedviews_base (1)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL] sanityCheckLoacked Error [incubator-gluten]

Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW closed issue #6528: [VL] sanityCheckLoacked Error
URL: https://github.com/apache/incubator-gluten/issues/6528


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org