You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Amit Jain (Jira)" <ji...@apache.org> on 2022/12/21 07:07:00 UTC
[jira] [Updated] (OAK-10048) DocumentStoreIndexerBase#buildFlatFileStore outputs the wrong path when FlatFileSplitter used
[ https://issues.apache.org/jira/browse/OAK-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amit Jain updated OAK-10048:
----------------------------
Description:
DocumentStoreIndexerBase#buildFlatFileStore below outputs the first FlatFileStore and its path which is incorrect if the FlatFileSplitter is triggered to split files based on config and index definitions.
Also, the buildFlatFileStoreList method always passes false to split but shoud instead use the configuration {{IndexerConfiguration.parallelIndexEnabled()}} to try splitting based on this configuration.
{code:java}
public FlatFileStore buildFlatFileStore() throws IOException, CommitFailedException {
NodeState checkpointedState = indexerSupport.retrieveNodeStateForCheckpoint();
Set<String> preferredPathElements = new HashSet<>();
Set<IndexDefinition> indexDefinitions = getIndexDefinitions();
for (IndexDefinition indexDf : indexDefinitions) {
preferredPathElements.addAll(indexDf.getRelativeNodeNames());
}
Predicate<String> predicate = s -> indexDefinitions.stream().anyMatch(indexDef -> indexDef.getPathFilter().filter(s) != PathFilter.Result.EXCLUDE);
FlatFileStore flatFileStore = buildFlatFileStoreList(checkpointedState, null, predicate, preferredPathElements, false, indexDefinitions).get(0);
log.info("FlatFileStore built at {}. To use this flatFileStore in a reindex step, set System Property-{} with value {}",
flatFileStore.getFlatFileStorePath(), OAK_INDEXER_SORTED_FILE_PATH, flatFileStore.getFlatFileStorePath());
return flatFileStore;
}
{code}
was:
DocumentStoreIndexerBase#buildFlatFileStore below outputs the first FlatFileStore and its path which is incorrect if the FlatFileSplitter is triggered to split files based on config and index definitions.
{code:java}
public FlatFileStore buildFlatFileStore() throws IOException, CommitFailedException {
NodeState checkpointedState = indexerSupport.retrieveNodeStateForCheckpoint();
Set<String> preferredPathElements = new HashSet<>();
Set<IndexDefinition> indexDefinitions = getIndexDefinitions();
for (IndexDefinition indexDf : indexDefinitions) {
preferredPathElements.addAll(indexDf.getRelativeNodeNames());
}
Predicate<String> predicate = s -> indexDefinitions.stream().anyMatch(indexDef -> indexDef.getPathFilter().filter(s) != PathFilter.Result.EXCLUDE);
FlatFileStore flatFileStore = buildFlatFileStoreList(checkpointedState, null, predicate, preferredPathElements, false, indexDefinitions).get(0);
log.info("FlatFileStore built at {}. To use this flatFileStore in a reindex step, set System Property-{} with value {}",
flatFileStore.getFlatFileStorePath(), OAK_INDEXER_SORTED_FILE_PATH, flatFileStore.getFlatFileStorePath());
return flatFileStore;
}
{code}
> DocumentStoreIndexerBase#buildFlatFileStore outputs the wrong path when FlatFileSplitter used
> ---------------------------------------------------------------------------------------------
>
> Key: OAK-10048
> URL: https://issues.apache.org/jira/browse/OAK-10048
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: oak-run
> Reporter: Amit Jain
> Priority: Major
>
> DocumentStoreIndexerBase#buildFlatFileStore below outputs the first FlatFileStore and its path which is incorrect if the FlatFileSplitter is triggered to split files based on config and index definitions.
> Also, the buildFlatFileStoreList method always passes false to split but shoud instead use the configuration {{IndexerConfiguration.parallelIndexEnabled()}} to try splitting based on this configuration.
> {code:java}
> public FlatFileStore buildFlatFileStore() throws IOException, CommitFailedException {
> NodeState checkpointedState = indexerSupport.retrieveNodeStateForCheckpoint();
> Set<String> preferredPathElements = new HashSet<>();
> Set<IndexDefinition> indexDefinitions = getIndexDefinitions();
> for (IndexDefinition indexDf : indexDefinitions) {
> preferredPathElements.addAll(indexDf.getRelativeNodeNames());
> }
> Predicate<String> predicate = s -> indexDefinitions.stream().anyMatch(indexDef -> indexDef.getPathFilter().filter(s) != PathFilter.Result.EXCLUDE);
> FlatFileStore flatFileStore = buildFlatFileStoreList(checkpointedState, null, predicate, preferredPathElements, false, indexDefinitions).get(0);
> log.info("FlatFileStore built at {}. To use this flatFileStore in a reindex step, set System Property-{} with value {}",
> flatFileStore.getFlatFileStorePath(), OAK_INDEXER_SORTED_FILE_PATH, flatFileStore.getFlatFileStorePath());
> return flatFileStore;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)