newbie questions

[Sagar Naik <sn...@attributor.com>]

hi 
I have few questions on hive and its use case.

1. hive-on-hadoop-20 accessing/processing data stored on hadoop-18-dfs
    The actual files are on hadoop-18 dfs and then I will create external table on hive-on-hadoop-20 with files pointing to hadoop-18-dfs.
    I don't think this is possible , given hadoop version incompatibility; but never hurts to ask


2. We download tons of urls and massage the data. The massaging goes thru various stages. We would like to monitor these stages 
   so I was thinking on doing a schema like following 
	One Table :
	
	url as STRING,
	
        massage_step1 is a STRUCT 
        massage_step2 is a STRUCT
	.
	.
	feature_set  is ARRAY<STRING>

      The STRUCT can have arrays on longs, ids, timestamps, success/failure, reasons


     Assuming tht I am correct track here :
	will I able to run queries like :
		q1. where massage_step1.reasons like '%Failed on fetching%'
		q2. where feature_set like 'shopping'
			(feature set is an array, I think I have to implement a UDFLike for Arrays)
		q3. where massage_step2.ids < 10K 

		q4. where count(*)  as count  where timestamps < 'SOME_DATE'  group by massage_step1.success = true
		
	In short , can I query on data in the complex types like Struct, Array, Map etc

3. Some of queries will require data from 2 or more structs and some wont. 
    In the above example, I keeping it one table (external table). The other option is multiple tables: one for each massage_step.
		
    In case of multiple tables, I will have to fire JOIN queries and in case of single table , I will filter data using where clause

   What is expensive: JOIN queries or filtering data using where clause ? 


Feedback is greatly appreciated

Thanks,
Sagar