Implement erasure coding as a layer on HDFS (HDFS-503)

[shravan kumar <sh...@gmail.com>]

Hi,

Can any one help me,

when i executed this patch(HDFS-503),with the help of start-raidnode.sh

i am getting this message.



STARTUP_MSG: Starting RaidNode
STARTUP_MSG:   host = hadoop/127.0.1.1
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.21.0



2011-04-29 01:14:56,447 INFO org.apache.hadoop.raid.RaidNode: RaidNode up
at: /127.0.0.1:60000
2011-04-29 01:14:56,448 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting

2011-04-29 01:14:56,455 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 60000: starting
2011-04-29 01:14:56,455 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 60000: starting
2011-04-29 01:14:56,461 INFO org.apache.hadoop.raid.RaidNode: *Triggering
Policy Filter shravan hdfs://localhost:9000/user/shravan*
2011-04-29 01:14:56,462 INFO org.apache.hadoop.raid.RaidNode: *Started
archive scan*
2011-04-29 01:14:56,495 INFO org.apache.hadoop.raid.RaidNode: *No filtered
paths for policy shravan*


I have data in the folder /user/shravan/(around 2 GB of data).
**

*My configurations*

*core-site.xml
*

*<configuration>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9000</value>
  </property>
</configuration>*

*hdfs-site.xml*

*<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>

<property>
<name>raid.config.file</name>
<value>/home/shashank/hadoop-0.21.0/conf/raid.xml</value>
<description>This is needed by the RaidNode </description>
</property>

<property>
<name>fs.hdfs.impl</name>
<value>org.apache.hadoop.hdfs.DistributedRaidFileSystem</value>
<description>The FileSystem for hdfs: uris.</description>
</property>

<property>
<name>hdfs.raid.locations</name>
<value>hdfs://localhost:9000/raid</value>
<description>The location for parity files. If this is
          is not defined, then defaults to /raid.</description>
</property>

<property>
<name>hdfs.raid.stripeLength</name>
<value>10</value>
<description>The number of blocks in a file to be combined into
a single raid parity block. The default value is 5. The lower
the number the greater is the disk space you will save when you
enable raid.</description>
</property>

<property>
<name>fs.raidnode.local</name>
<value>true</value>
<description>If you do not want to use your map-reduce cluster to
raid files in parallel, then specify "true". By default, this
value is false, which means that the RaidNode uses the default
map-reduce cluster to generate parity blocks.</description>
</property>

<property>
<name>raid.policy.rescan.interval</name>
<value>500</value>
<description>Specify the periodicity in milliseconds after which
all source paths are rescanned and parity blocks recomputed if
necessary. By default, this value is 1 hour.</description>
</property>

</configuration>*


**

*mapred-site.xml*

*<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:9001</value>
  </property>
</configuration>*

*raid.xml*
**

*   <configuration>
    <srcPath prefix="hdfs://localhost:9000/user/shravan/">
      <policy name = "shravan">
        <property>
          <name>srcReplication</name>
          <value>3</value>
          <description> pick files for RAID only if their replication factor
is
                        greater than or equal to this value.
          </description>
        </property>
        <property>
          <name>targetReplication</name>
          <value>2</value>
          <description> after RAIDing, decrease the replication factor of a
file to
                        this value.
          </description>
        </property>
        <property>
          <name>metaReplication</name>
          <value>2</value>
          <description> the replication factor of the RAID meta file
          </description>
        </property>
        <property>
          <name>modTimePeriod</name>
          <value>100</value>
          <description> time (milliseconds) after a file is modified to make
it a
                        candidate for RAIDing
          </description>
        </property>
      </policy>
    </srcPath>
   </configuration>

*




Thank you in advance*
*


-- 
Regards,
M.Shravankumar,**




-- 
Regards,
M.Shravankumar,
*Looking for job in DELHI,HYDERABAD,*
ME(SE),
PSG TECH,
Coimbatore,
09042031485.