You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spot.apache.org by le...@apache.org on 2017/07/11 22:54:07 UTC

incubator-spot git commit: Removing Spot 0.9 references from documentation

Repository: incubator-spot
Updated Branches:
  refs/heads/SPOT-35_graphql_api 8e1926d2f -> 061530d6a


Removing Spot 0.9 references from documentation


Project: http://git-wip-us.apache.org/repos/asf/incubator-spot/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spot/commit/061530d6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spot/tree/061530d6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spot/diff/061530d6

Branch: refs/heads/SPOT-35_graphql_api
Commit: 061530d6a55317b681aa5de983194565d9b9bd24
Parents: 8e1926d
Author: Moy Valdovinos <mo...@gmail.com>
Authored: Tue Jul 11 16:08:29 2017 -0500
Committer: LedaLima <ga...@intel.com>
Committed: Tue Jul 11 17:53:51 2017 -0500

----------------------------------------------------------------------
 spot-setup/README.md             |  2 +-
 spot-setup/migration/README.md   |  8 ++++----
 spot-setup/migration/__init__.py | 17 ++++++++++++++++-
 3 files changed, 21 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-spot/blob/061530d6/spot-setup/README.md
----------------------------------------------------------------------
diff --git a/spot-setup/README.md b/spot-setup/README.md
index 2b803ab..1d486a6 100644
--- a/spot-setup/README.md
+++ b/spot-setup/README.md
@@ -72,7 +72,7 @@ To get to parquet format we need a staging table to store CSV data temporarily f
 - proxy_timeline
 
 
-## Migrating OA Data from Spot 0.9 to 1.0
+## Migrating OA Data to Spot 1.0
 
 Please review migration documentation [here](migration/README.md).
 

http://git-wip-us.apache.org/repos/asf/incubator-spot/blob/061530d6/spot-setup/migration/README.md
----------------------------------------------------------------------
diff --git a/spot-setup/migration/README.md b/spot-setup/migration/README.md
index f0f45a4..29314ef 100644
--- a/spot-setup/migration/README.md
+++ b/spot-setup/migration/README.md
@@ -1,6 +1,6 @@
-## OA Data Migration from Spot 0.9 to Spot 1.0
+## OA Data Migration to Spot 1.0
 
-This document is intended for any developer or sysadmin who wants to migrate their existing OA data from Spot 0.9 to Spot 1.0. In Spot 0.9, OA data was stored in CSV files in a given location in the server used for OA (specified in spot.conf during original installation). In Spot 1.0, OA data is stored in Impala tables. The purposes of these scripts are to migrate independently each use case (flow, proxy and dns) from those CSV files into the new Impala tables.
+This document is intended for any developer or sysadmin who wants to migrate their existing OA data to Spot 1.0. In previous Spot releases, OA data was stored in CSV files in a given location in the server used for OA (specified in spot.conf during original installation). In Spot 1.0, OA data is stored in Impala tables. The purposes of these scripts are to migrate independently each use case (flow, proxy and dns) from those CSV files into the new Impala tables.
 
 This migration process is optional and only for those users who want to keep usable their OA data (scores, edges, chords, dendros, ingest summaries, storyboards, threat investigations and timelines) generated in the previous Spot version. 
 
@@ -49,7 +49,7 @@ timeline-*.csv | proxy_timeline
 
 ### Data Flow
 
-There is a launch and single script that will migrate all specified pipelines. This process will read each of the CSV from the existing location and import data to Impala tables accordingly, creating first a staging database and tables to load the records in the CSV and then insert that data into the new Spot 1.0 tables. You must execute this migration process from the server where Spot 0.9 CSV files are located. You may provide one pipeline or all (flow, dns and proxy) according to your needs and your existing data. At the end of each script, the old data pipeline folder will be moved from the original location to a backup folder. Staging tables and their respective HDFS paths will be removed.
+There is a launch and single script that will migrate all specified pipelines. This process will read each of the CSV from the existing location and import data to Impala tables accordingly, creating first a staging database and tables to load the records in the CSV and then insert that data into the new Spot 1.0 tables. You must execute this migration process from the server where old Spot release is located. You may provide one pipeline or all (flow, dns and proxy) according to your needs and your existing data. At the end of each script, the old data pipeline folder will be moved from the original location to a backup folder. Staging tables and their respective HDFS paths will be removed.
 
 ### Execution
 
@@ -59,7 +59,7 @@ There is a launch and single script that will migrate all specified pipelines. T
 
 where variables mean:
 - **PIPELINES** - Comma-separated list of the pipelines to be migrated 
-- **OLD_OA_PATH** - Path to the old Spot-OA (0.9 release) directory in the local filesystem 
+- **OLD_OA_PATH** - Path to the old Spot-OA release directory in the local filesystem 
 - **STAGING_DB_NAME** - Name of the staging database to be created to temporarily store these records
 - **STAGING_DB_HDFS_PATH** - HDFS path of the staging database to be created to temporarily store these records
 - **NEW_SPOT_IMPALA_DB** - Database name of the Spot 1.0 Impala tables. Use the same as in the spot.conf when the new Spot release was installed 

http://git-wip-us.apache.org/repos/asf/incubator-spot/blob/061530d6/spot-setup/migration/__init__.py
----------------------------------------------------------------------
diff --git a/spot-setup/migration/__init__.py b/spot-setup/migration/__init__.py
index 8b13789..ecb1860 100644
--- a/spot-setup/migration/__init__.py
+++ b/spot-setup/migration/__init__.py
@@ -1 +1,16 @@
-
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
\ No newline at end of file