Change in asterixdb[master]: [ASTERIXDB-2194][COMP] Introduce datasource functions

["abdullah alamoudi (Code Review)" <do...@asterixdb.incubator.apache.org>]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/2216

to look at the new patch set (#6).

Change subject: [ASTERIXDB-2194][COMP] Introduce datasource functions
......................................................................

[ASTERIXDB-2194][COMP] Introduce datasource functions

- user model changes: yes
  Some functions can be datasources
- storage format changes: no
- interface changes: yes
  - Add IDatasourceFunction: A function that is also a datasource
  - Add IFunctionToDataSourceTransformer: transform an unnest
    function into a datascan during compilation

Details:
- Currently, functions are location agnostic and are run on
  parameters that are either passed through them during compile
  time or runtime.
- An exception to this is the dataset function which has
  an associated location constraints running on the nodes
  which host the dataset.
- In this change, we introduce a general framework that allows
  creation of new functions similar to the dataset function.
- Such functions are called datasource Functions.
- A datasource function takes constant parameters and run on
  a set of partitions similar to the dataset function.
- The first example of such functions is the DatasetResources
  function.
- The DatasetResources function takes two parameters, a dataverse
  and a dataset. It is then run on all nodes and returns a set
  of dataset resources.
- Test cases are added for this function.

Change-Id: Ibcf807ac713a21e8f4d59868525467386e801303
---
M asterixdb/asterix-algebra/pom.xml
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/AnalysisUtil.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/MetaFunctionToMetaVariableRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroducePrimaryIndexForAggregationRule.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetComponentsDatasource.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetComponentsFunction.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetComponentsReader.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetComponentsTransformer.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetResourcesDatasource.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetResourcesFunction.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetResourcesReader.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetResourcesTransformer.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/DatasetTransformer.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/FeedTransformer.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/FunctionReader.java
A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/util/FunctionTransformer.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
A asterixdb/asterix-app/src/main/java/org/apache/asterix/util/MetadataBuiltinFunctions.java
M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.1.ddl.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.2.update.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset-resources/dataset-resources.3.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset-resources/dataset-resources.4.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset-resources/dataset-resources.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedConstants.java
A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IDatasourceFunction.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBuiltinEntities.java
A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AbstractDatasourceFunction.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/DataSource.java
A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/FunctionDataSource.java
A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/FunctionDataSourceFactory.java
D asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/MetadataBuiltinFunctions.java
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/IFunctionToDataSourceTransformer.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/ARecordType.java
43 files changed, 1,596 insertions(+), 465 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/16/2216/6
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2216
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibcf807ac713a21e8f4d59868525467386e801303
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <ba...@gmail.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>