Re: Review Request 73160: ATLAS-4112 : Basic Search : Incorrect results when searching with value having special char and contains/beginsWith/endsWith operator

[Sarath Subramanian <sa...@apache.org>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73160/#review222539
-----------------------------------------------------------




repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Line 461 (original), 462 (patched)
<https://reviews.apache.org/r/73160/#comment311645>

    consider refactoring this method for better readability:
    
        private boolean isIndexSearchable(FilterCriteria filterCriteria, AtlasStructType structType) throws AtlasBaseException {
            String      attributeName  = filterCriteria.getAttributeName();
            String      attributeValue = filterCriteria.getAttributeValue();
            AtlasType   attributeType  = structType.getAttributeType(attributeName);
            String      typeName       = attributeType.getTypeName();
            String      qualifiedName  = structType.getVertexPropertyName(attributeName);
            Set<String> indexedKeys    = context.getIndexedKeys();
            boolean     ret            = indexedKeys != null && indexedKeys.contains(qualifiedName);
    
            SearchParameters.Operator                  operator  = filterCriteria.getOperator();
            AtlasStructDef.AtlasAttributeDef.IndexType indexType = structType.getAttributeDef(attributeName).getIndexType();
    
            if (ret) { // index exists
                // for string type attributes, don't use index query in the following cases:
                //   - operation is NEQ, as it might return fewer entries due to tokenization of vertex property value
                //   - value-to-compare has special characters
                if (AtlasBaseTypeDef.ATLAS_TYPE_STRING.equals(typeName)) {
                    if (operator == SearchParameters.Operator.NEQ || operator == SearchParameters.Operator.NOT_CONTAINS) {
                        if (LOG.isDebugEnabled()) {
                            LOG.debug("{} operator found for string attribute {}, deferring to in-memory or graph query (might cause poor performance)", operator, qualifiedName);
                        }
    
                        ret = false;
                    } else if (operator == SearchParameters.Operator.CONTAINS && AtlasAttribute.hastokenizeChar(attributeValue) && indexType == null) {
                        if (LOG.isDebugEnabled()) {
                            LOG.debug("{} operator found for string (TEXT) attribute {} and special characters found in filter value {}, deferring to in-memory or graph query (might cause poor performance)", attributeValue);
                        }
    
                        ret = false;
                    }
                }
            }
    
            if (LOG.isDebugEnabled()) {
                if (!ret) {
                    LOG.debug("Not using index query for: attribute='{}', operator='{}', value='{}'", qualifiedName, operator, attributeValue);
                }
            }
    
            return ret;
        }



repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
Lines 32 (patched)
<https://reviews.apache.org/r/73160/#comment311644>

    nit: review/remove unused imports


- Sarath Subramanian


On Jan. 27, 2021, 8:56 p.m., Pinal Shah wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73160/
> -----------------------------------------------------------
> 
> (Updated Jan. 27, 2021, 8:56 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-4112
>     https://issues.apache.org/jira/browse/ATLAS-4112
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Problem:** While searching with wildcard search i.e contains, beginsWith, endsWith operator, which has escape characters in search attribute value gives incorrect results
> 
> **Example:** While searching qualifiedName beginsWith "default.datagen_table_sensitive_725_1."
> 
> **Solution:** Remove quotes and wildcard '*' from query
> 
> **Steps to reproduce:**
> 1. Create a hive_table with name ends with a digit: datagen_table_sensitive_725_1
> 2. Search By Type: hive_columns and select attribute filter: qualifiedName as "default.datagen_table_sensitive_725_1." shows no Records found but if qualifiedName is changed to "default.datagen_table_sensitive_725_1" it shows results with qualifiedName of column "default.datagen_table_sensitive_725_1.col1@cm"
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 050980994 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 01daf53d2 
>   repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java 0da60d37f 
> 
> 
> Diff: https://reviews.apache.org/r/73160/diff/2/
> 
> 
> Testing
> -------
> 
> Added unit test cases
> Precommit: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/353/console
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>