You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "qidaye (via GitHub)" <gi...@apache.org> on 2024/01/02 10:01:17 UTC
[I] [Bug] can not get long text content from ES catalog [doris]
qidaye opened a new issue, #29417:
URL: https://github.com/apache/doris/issues/29417
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
master
### What's Wrong?
```sql
mysql> select * from test;
+---------+--------------------+---------+------------+------------------------+---------------------+---------------+
| content | id | merCode | merOrderId | orderCode | timeStr | timestamp |
+---------+--------------------+---------+------------+------------------------+---------------------+---------------+
| NULL | 715961032309346304 | 600192 | * | 6001922392716133188467 | 2023-09-27 16:13:32 | 1695802412009 |
+---------+--------------------+---------+------------+------------------------+---------------------+---------------+
1 row in set (0.02 sec)
```
### What You Expected?
```sql
select * from test;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------+------------+------------------------+---------------------+---------------+
| content
| id | merCode | merOrderId | orderCode | timeStr | timestamp |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------+------------+------------------------+---------------------+---------------+
| [<font color=blue>v3 riskCtrl - Send</font>] : <br/>{"txnType":"SALE","merchantNo":"600192","risk3dsStrategy":"DEFAULT","payerCardInfo":{"index":null,"cardHash":null,"channelSignId":null,"cardBin":"4211023320","cardNumber":"421102****0023","vaAccount":null,"cvv":"***","month":"***","year":"***","holderName":"Francisco José Guzmán Castro","cardBrand":"VISA","cardCountry":"ES","bankNo":null,"bankName":null,"idNo":null},"merchantChannel":"API","paymentProductType":"DIRECT","productType":"CARD","pricingAmount":{"amount":"*","currency":"EUR"},"orderMsg":{"appId":null,"website":"www.jessemade.es","pacypayReturnUrl":"https://www.jessemade.es/api/payment/return?order_id=*&payment_channel=pacypay","returnUrl":"https://www.jessemade.es/api/payment/return?order_id=*&payment_channel=pacypay","products":"Una tarjeta-A mi novio/novia,Llavero para pareja mano a mano 2 letras personalizadas con fecha,","billingInformation":{"firstName":"Francisco José","lastName":"Guzmán Castro","phone":"+
*","email":"*@gmail.com","postalCode":"*","address":"Plaça Joan Serrats n°7, 3° - 2°","country":"ES","province":"Barcelona [Barcelona]","city":"Martorell","street":null,"number":null,"identityNumber":null,"birthDate":null,"jpFirstName":null,"jpLastName":null},"shippingInformation":{"firstName":null,"lastName":null,"phone":"+*","email":"*@gmail.com","postalCode":"08760","address":"Plaça Joan Serrats n°7, 3° - 2°","country":"ES","province":"Barcelona [Barcelona]","city":"Martorell","street":null,"number":null,"identityNumber":null,"birthDate":null,"jpFirstName":null,"jpLastName":null},"deviceInfo":{"javaEnabled":"false","jsEnabled":null,"colorDepth":"24","screenHeight":"800","screenWidth":"360","timeZoneOffset":"-120","accept":"*/*","userAgent":"Mozilla/5.0 (Linux; Android 13; SM-A536B Build/TP1A.220624.014; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/117.0.0.0 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/433.0.0.31.111;]","contentLength":"12449","language":"en",
"ip":"*","sessionId":null}},"originalTxnId":null,"externalOrderId":"*","externalRequestId":null,"externalTime":null,"externalTimeZone":null,"platform":null,"amountUSD":null,"settleMoney":null,"requestType":null,"requestId":null,"systemSource":null,"traceId":null,"serviceId":"*","serviceType":null} | 715961032309346304 | 600192 | * | 6001922392716133188467 | 2023-09-27 16:13:32 | 1695802412009 |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------+------------+------------------------+---------------------+---------------+
1 row in set (0.02 sec)
```
### How to Reproduce?
1. index a doc
```bash
curl -sXPUT localhost:29200/test/_doc/1?pretty -H 'Content-Type:application/json' -d'{
> "id": 715961032309346304,
> "merCode": "600192",
> "merOrderId": "*",
> "orderCode": "6001922392716133188467",
> "content": "[<font color='blue'>v3 riskCtrl - Send</font>] : <br/>{\"txnType\":\"SALE\",\"merchantNo\":\"600192\",\"risk3dsStrategy\":\"DEFAULT\",\"payerCardInfo\":{\"index\":null,\"cardHash\":null,\"channelSignId\":null,\"cardBin\":\"4211023320\",\"cardNumber\":\"421102****0023\",\"vaAccount\":null,\"cvv\":\"***\",\"month\":\"***\",\"year\":\"***\",\"holderName\":\"Francisco José Guzmán Castro\",\"cardBrand\":\"VISA\",\"cardCountry\":\"ES\",\"bankNo\":null,\"bankName\":null,\"idNo\":null},\"merchantChannel\":\"API\",\"paymentProductType\":\"DIRECT\",\"productType\":\"CARD\",\"pricingAmount\":{\"amount\":\"*\",\"currency\":\"EUR\"},\"orderMsg\":{\"appId\":null,\"website\":\"www.jessemade.es\",\"pacypayReturnUrl\":\"https://www.jessemade.es/api/payment/return?order_id=*&payment_channel=pacypay\",\"returnUrl\":\"https://www.jessemade.es/api/payment/return?order_id=*&payment_channel=pacypay\",\"products\":\"Una tarjeta-A mi novio/novia,Llavero para pareja mano a mano 2 letr
as personalizadas con fecha,\",\"billingInformation\":{\"firstName\":\"Francisco José\",\"lastName\":\"Guzmán Castro\",\"phone\":\"+*\",\"email\":\"*@gmail.com\",\"postalCode\":\"*\",\"address\":\"Plaça Joan Serrats n°7, 3° - 2°\",\"country\":\"ES\",\"province\":\"Barcelona [Barcelona]\",\"city\":\"Martorell\",\"street\":null,\"number\":null,\"identityNumber\":null,\"birthDate\":null,\"jpFirstName\":null,\"jpLastName\":null},\"shippingInformation\":{\"firstName\":null,\"lastName\":null,\"phone\":\"+*\",\"email\":\"*@gmail.com\",\"postalCode\":\"08760\",\"address\":\"Plaça Joan Serrats n°7, 3° - 2°\",\"country\":\"ES\",\"province\":\"Barcelona [Barcelona]\",\"city\":\"Martorell\",\"street\":null,\"number\":null,\"identityNumber\":null,\"birthDate\":null,\"jpFirstName\":null,\"jpLastName\":null},\"deviceInfo\":{\"javaEnabled\":\"false\",\"jsEnabled\":null,\"colorDepth\":\"24\",\"screenHeight\":\"800\",\"screenWidth\":\"360\",\"timeZoneOffset\":\"-120\",\"accept\":\"*/*\",\
"userAgent\":\"Mozilla/5.0 (Linux; Android 13; SM-A536B Build/TP1A.220624.014; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/117.0.0.0 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/433.0.0.31.111;]\",\"contentLength\":\"12449\",\"language\":\"en\",\"ip\":\"*\",\"sessionId\":null}},\"originalTxnId\":null,\"externalOrderId\":\"*\",\"externalRequestId\":null,\"externalTime\":null,\"externalTimeZone\":null,\"platform\":null,\"amountUSD\":null,\"settleMoney\":null,\"requestType\":null,\"requestId\":null,\"systemSource\":null,\"traceId\":null,\"serviceId\":\"*\",\"serviceType\":null}",
> "timeStr": "2023-09-27 16:13:32",
> "timestamp": 1695802412009
> }
> '
```
2. get mapping
```bash
curl localhost:29200/test/_mapping?pretty
{
"test" : {
"mappings" : {
"properties" : {
"content" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
},
"merCode" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"merOrderId" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"orderCode" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"timeStr" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"timestamp" : {
"type" : "long"
}
}
}
}
}
```
3. create catalog in Doris
```sql
CREATE CATALOG `test_es_query_es7` PROPERTIES (
"type" = "es",
"nodes_discovery" = "false",
"hosts" = "http://127.0.0.1:29200",
"enable_keyword_sniff" = "false",
"enable_docvalue_scan" = "true",
"create_time" = "2023-12-31 09:02:37.043"
);
```
4. select in Doris
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] can not get long text content from ES catalog [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye closed issue #29417: [Bug] can not get long text content from ES catalog
URL: https://github.com/apache/doris/issues/29417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org