You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Attila Magyar <am...@hortonworks.com> on 2018/01/02 12:05:11 UTC
Re: Review Request 64852: Ambari Infra Manager: deleting documents

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64852/#review194624
-----------------------------------------------------------


Ship it!




Ship It!

- Attila Magyar


On Dec. 28, 2017, 2:09 p.m., Krisztian Kasa wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64852/
> -----------------------------------------------------------
> 
> (Updated Dec. 28, 2017, 2:09 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Oliver Szabo, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22653 and AMBARI-22702
>     https://issues.apache.org/jira/browse/AMBARI-22653
>     https://issues.apache.org/jira/browse/AMBARI-22702
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> * defining new job for deleting documents from solr: collection and filter field can be defined in properties
> * archiving job supports deleting the succesfully uploaded documents
> * date time format in output filename can be defined in property file
> * s3 secret and access keys can be defined in a csv file and this file can be referenced from infra-manager.properties or in environmental variables
> * generic property processing for jobs
> * Helper class for processing parametrized solr query text
> * new IT case: Running archiving job with a bigger start value than end value exports and deletes 0 documents
> * new IT case: After Deleting job deletes documents from solr no document found in the specified interval
> * existing IT cases for archiving also check that solr has no more documents left in the specified interval after archiving job finished
> 
> 
> Diffs
> -----
> 
>   ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/OffsetDateTimeConverter.java 9db562cec3 
>   ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/steps/AbstractInfraSteps.java 703e1cfa67 
>   ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/steps/ExportJobsSteps.java 4a09d7d7ee 
>   ambari-infra/ambari-infra-manager-it/src/test/resources/stories/infra_api_tests.story cd1f49d052 
>   ambari-infra/ambari-infra-manager/docker/infra-manager-docker-compose.sh ab026598b1 
>   ambari-infra/ambari-infra-manager/pom.xml 67bf7d133b 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/JobProperties.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/PropertyMap.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/SolrDAOBase.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/AbstractFileAction.java 7a30393224 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/CompositeFileAction.java 8421802cd7 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/Document.java 1f3957a034 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportConfiguration.java 189591102b 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportJobListener.java 3b6c402458 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportProperties.java 37f6d1b669 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportPropertyMap.java 9af4afc630 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentWiper.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/FileAction.java d3f2a65e07 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/FileNameSuffixFormatter.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/ItemWriterListener.java 742777109e 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/LocalDocumentItemWriter.java baad61bdcb 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/LocalItemWriterListener.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/S3Properties.java 0979f10bfc 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/S3Uploader.java deeb9c7ea0 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrDAO.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrDocumentIterator.java 2e7341d365 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrDocumentSource.java 5ded9acf85 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrParametrizedString.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrProperties.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/SolrQueryBuilder.java b3ea14ed65 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/TarGzCompressor.java 55ba58a0f2 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/WriteCompletedEvent.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/deleting/DocumentDeletingConfiguration.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/deleting/DocumentDeletingProperties.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/deleting/DocumentDeletingPropertyMap.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/deleting/DocumentWiperTasklet.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/main/resources/infra-manager.properties 27b36b36fc 
>   ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/archive/DocumentExportPropertiesTest.java ae93710b2d 
>   ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/archive/FileNameSuffixFormatterTest.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/archive/LocalDocumentItemWriterTest.java 3af93bc25d 
>   ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/archive/SolrParametrizedStringTest.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/archive/SolrPropertiesTest.java PRE-CREATION 
>   ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/archive/SolrQueryBuilderTest.java e9513dccd9 
> 
> 
> Diff: https://reviews.apache.org/r/64852/diff/3/
> 
> 
> Testing
> -------
> 
> * unit tests,
> * integration tests using docker containers
> * manual testing using docker containers and ide
> 
> 
> Thanks,
> 
> Krisztian Kasa
> 
>