You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Izek Greenfield (Jira)" <ji...@apache.org> on 2020/02/17 08:29:00 UTC
[jira] [Updated] (SPARK-30332) When running sql query with limit
catalyst throw StackOverFlow exception
[ https://issues.apache.org/jira/browse/SPARK-30332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Izek Greenfield updated SPARK-30332:
------------------------------------
Attachment: AGGR_41418.csv
AGGR_41410.csv
AGGR_41406.csv
AGGR_41390.csv
AGGR_41380.csv
PORTFOLIO_41446.csv
> When running sql query with limit catalyst throw StackOverFlow exception
> -------------------------------------------------------------------------
>
> Key: SPARK-30332
> URL: https://issues.apache.org/jira/browse/SPARK-30332
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Environment: spark version 3.0.0-preview
> Reporter: Izek Greenfield
> Priority: Major
> Attachments: AGGR_41380.csv, AGGR_41390.csv, AGGR_41406.csv, AGGR_41410.csv, AGGR_41418.csv, PORTFOLIO_41446.csv, T_41233.csv
>
>
> Running that SQL:
> {code:sql}
> SELECT BT_capital.asof_date,
> BT_capital.run_id,
> BT_capital.v,
> BT_capital.id,
> BT_capital.entity,
> BT_capital.level_1,
> BT_capital.level_2,
> BT_capital.level_3,
> BT_capital.level_4,
> BT_capital.level_5,
> BT_capital.level_6,
> BT_capital.path_bt_capital,
> BT_capital.line_item,
> t0.target_line_item,
> t0.line_description,
> BT_capital.col_item,
> BT_capital.rep_amount,
> root.orgUnitId,
> root.cptyId,
> root.instId,
> root.startDate,
> root.maturityDate,
> root.amount,
> root.nominalAmount,
> root.quantity,
> root.lkupAssetLiability,
> root.lkupCurrency,
> root.lkupProdType,
> root.interestResetDate,
> root.interestResetTerm,
> root.noticePeriod,
> root.historicCostAmount,
> root.dueDate,
> root.lkupResidence,
> root.lkupCountryOfUltimateRisk,
> root.lkupSector,
> root.lkupIndustry,
> root.lkupAccountingPortfolioType,
> root.lkupLoanDepositTerm,
> root.lkupFixedFloating,
> root.lkupCollateralType,
> root.lkupRiskType,
> root.lkupEligibleRefinancing,
> root.lkupHedging,
> root.lkupIsOwnIssued,
> root.lkupIsSubordinated,
> root.lkupIsQuoted,
> root.lkupIsSecuritised,
> root.lkupIsSecuritisedServiced,
> root.lkupIsSyndicated,
> root.lkupIsDeRecognised,
> root.lkupIsRenegotiated,
> root.lkupIsTransferable,
> root.lkupIsNewBusiness,
> root.lkupIsFiduciary,
> root.lkupIsNonPerforming,
> root.lkupIsInterGroup,
> root.lkupIsIntraGroup,
> root.lkupIsRediscounted,
> root.lkupIsCollateral,
> root.lkupIsExercised,
> root.lkupIsImpaired,
> root.facilityId,
> root.lkupIsOTC,
> root.lkupIsDefaulted,
> root.lkupIsSavingsPosition,
> root.lkupIsForborne,
> root.lkupIsDebtRestructuringLoan,
> root.interestRateAAR,
> root.interestRateAPRC,
> root.custom1,
> root.custom2,
> root.custom3,
> root.lkupSecuritisationType,
> root.lkupIsCashPooling,
> root.lkupIsEquityParticipationGTE10,
> root.lkupIsConvertible,
> root.lkupEconomicHedge,
> root.lkupIsNonCurrHeldForSale,
> root.lkupIsEmbeddedDerivative,
> root.lkupLoanPurpose,
> root.lkupRegulated,
> root.lkupRepaymentType,
> root.glAccount,
> root.lkupIsRecourse,
> root.lkupIsNotFullyGuaranteed,
> root.lkupImpairmentStage,
> root.lkupIsEntireAmountWrittenOff,
> root.lkupIsLowCreditRisk,
> root.lkupIsOBSWithinIFRS9,
> root.lkupIsUnderSpecialSurveillance,
> root.lkupProtection,
> root.lkupIsGeneralAllowance,
> root.lkupSectorUltimateRisk,
> root.cptyOrgUnitId,
> root.name,
> root.lkupNationality,
> root.lkupSize,
> root.lkupIsSPV,
> root.lkupIsCentralCounterparty,
> root.lkupIsMMRMFI,
> root.lkupIsKeyManagement,
> root.lkupIsOtherRelatedParty,
> root.lkupResidenceProvince,
> root.lkupIsTradingBook,
> root.entityHierarchy_entityId,
> root.entityHierarchy_Residence,
> root.lkupLocalCurrency,
> root.cpty_entityhierarchy_entityId,
> root.lkupRelationship,
> root.cpty_lkupRelationship,
> root.entityNationality,
> root.lkupRepCurrency,
> root.startDateFinancialYear,
> root.numEmployees,
> root.numEmployeesTotal,
> root.collateralAmount,
> root.guaranteeAmount,
> root.impairmentSpecificIndividual,
> root.impairmentSpecificCollective,
> root.impairmentGeneral,
> root.creditRiskAmount,
> root.provisionSpecificIndividual,
> root.provisionSpecificCollective,
> root.provisionGeneral,
> root.writeOffAmount,
> root.interest,
> root.fairValueAmount,
> root.grossCarryingAmount,
> root.carryingAmount,
> root.code,
> root.lkupInstrumentType,
> root.price,
> root.amountAtIssue,
> root.yield,
> root.totalFacilityAmount,
> root.facility_rate,
> root.spec_indiv_est,
> root.spec_coll_est,
> root.coll_inc_loss,
> root.impairment_amount,
> root.provision_amount,
> root.accumulated_impairment,
> root.exclusionFlag,
> root.lkupIsHoldingCompany,
> root.instrument_startDate,
> root.entityResidence,
> fxRate.enumerator,
> fxRate.lkupFromCurrency,
> fxRate.rate,
> fxRate.custom1,
> fxRate.custom2,
> fxRate.custom3,
> GB_position.lkupIsECGDGuaranteed,
> GB_position.lkupIsMultiAcctOffsetMortgage,
> GB_position.lkupIsIndexLinked,
> GB_position.lkupIsRetail,
> GB_position.lkupCollateralLocation,
> GB_position.percentAboveBBR,
> GB_position.lkupIsMoreInArrears,
> GB_position.lkupIsArrearsCapitalised,
> GB_position.lkupCollateralPossession,
> GB_position.lkupIsLifetimeMortgage,
> GB_position.lkupLoanConcessionType,
> GB_position.lkupIsMultiCurrency,
> GB_position.lkupIsJointIncomeBasis,
> GB_position.ratioIncomeMultiple,
> GB_position.interestRate,
> GB_position.exclusionFlag,
> GB_position.lkupFDIDirection,
> GB_position.lkupIsRTGS,
> GB_positionExtended.nonRecourseFinanceAmount,
> GB_positionExtended.arrearsAmount,
> GB_Counterparty.lkupIsClearingFirm,
> GB_Counterparty.lkupIsIntermediateFinCorp,
> GB_Counterparty.lkupIsImpairedCreditHistory,
> GB_Counterparty.lkupFDIRelationship FROM portfolio_41446 BT_capital
> JOIN aggr_41390 root ON (root.id = BT_capital.id AND root.entity = BT_capital.entity AND (root.instance_id = 'e3b82807-9371-44f4-9c97-d63cde21dbf2'))
> JOIN aggr_41380 fxRate ON (fxRate.lkupToCurrency = root.lkupRepCurrency AND (root.lkupCurrency = fxRate.lkupFromCurrency) AND (fxRate.instance_id = '641f3e44-8d0a-4040-8f26-1d459c5a47a3'))
> LEFT OUTER JOIN aggr_41418 GB_position ON (GB_position.id = root.id AND (GB_position.instance_id = 'a8eb71eb-906b-4b76-a977-589618f5f1b5'))
> LEFT OUTER JOIN aggr_41406 GB_positionExtended ON (GB_positionExtended.id = GB_position.id AND (GB_positionExtended.instance_id = 'bc7415ca-8cda-4c64-a323-fdfe72b00d54'))
> LEFT OUTER JOIN aggr_41410 GB_Counterparty ON (GB_Counterparty.id = root.cptyId AND (GB_Counterparty.instance_id = '6e0d1229-c142-4411-81e2-c1cc5f636843'))
> LEFT OUTER JOIN t_41233 t0 ON (t0.line_item = BT_capital.line_item)
> WHERE ((BT_capital.instance_id = '4fa9a3a5-67e2-4c78-9708-76849d424123')) AND (root.exclusionFlag!='yes'
> ) AND (GB_position.exclusionFlag!='yes')
> AND
> ((((NVL(t0.target_line_item,BT_capital.line_item) = '10') AND
> (BT_capital.col_item = 'Euro') AND
> (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '10') AND
> (BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '10') AND
> (BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '11A') AND (BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '11A') AND (BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '11A') AND (BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '11B') AND (BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '11B') AND (BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '11B') AND (BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19A') AND (BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19A') AND (BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19A') AND (BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19B') AND (BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19B') AND (BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19B') AND (BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19BL') AND (BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19BL') AND (BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19BL') AND (BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19CCA') AND (BT_capital.col_item = 'Euro') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19CCA') AND (BT_capital.col_item = 'Other currencies') AND (BT_capital.entity = 'E002')) OR
> ((NVL(t0.target_line_item,BT_capital.line_item) = '19CCA') AND (BT_capital.col_item = 'Sterling') AND (BT_capital.entity = 'E002'))) AND (BT_capital.level_1 != 'Orphans')) limit 101
> {code}
> I get a StackOverflow exception. remove the LIMIT at the end and all is working.
> with spark version 2.4.4 it does not happen
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org