Re: need help for setting muti-node hbase on hadoop

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: need help for setting muti-node hbase on hadoop

晋光峰
>   Dears,
>
> now i'm try to using hbase on hadoop. I have developed the source code
> running on single machine, and hbase/hadoop working fine.
>
> now I try to move the program to mutiple node. The machines are organized
> like this:
>     machine 1: act as namenode and jobtracker
>     machine 2: act as datanode and tasktracker
>     machine 3 and machine 4 act same as machine2, as data node and
> tasktracker.
>
> I have set up the hadoop framework successfully on these 4 machines. Now i
> want to install hbase also on these 4 machines. I have read the HBase 0.1.2
> API front page about how to configure the muti-node hbase. But still not
> very clear about the description. I want to install hbase master node on
> machine1, and machine2, 3 and 4 for region nodes.
>
> So my questions are:
>     1, Do i need to install hbase software on all the four machines?
>     2, Is the configuration parameters in hbase master node are same as
> hbase region server node?
>     3, Is hbase-0.1.2 support lates hadoop-0.17.0?
>
> I have tried to install on the four machines(hbase0.1.2 with
> hadoop0.17.0). But it failed. the configureations are listed below:
> for hbase-site.xml:
>  <configuration>
>   <property>
>     <name>hbase.master</name>
>     <value>machine1:60000</value>
>     <description>The host and port that the HBase master runs at.
>     </description>
>   </property>
>
>   <property>
>     <name>hbase.rootdir</name>
>     <value>hdfs://machine1:9000/hbase</value>
>     <description>The directory shared by region servers.
>     </description>
>   </property>
>
> </configuration>
>
> for regionserver file:
>     machine2
>     machine3
>     machine4
>
> The hbase are installed on all the four machines and configuration
> parameters are the same
>
> Any one can help me to clarify the questions?
>
> Thanks & Regards
> Guangfeng
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: need help for setting muti-node hbase on hadoop

stack-3
晋光峰 wrote:
>> So my questions are:
>>     1, Do i need to install hbase software on all the four machines?
>>    
Yes.

>>     2, Is the configuration parameters in hbase master node are same as
>> hbase region server node?
>>    
Yes.

>>     3, Is hbase-0.1.2 support lates hadoop-0.17.0?
>>    

No. HBase runs on Hadoop 0.16.x. The next hbase release, 0.2.0, will
require Hadoop 0.17.

>> I have tried to install on the four machines(hbase0.1.2 with
>> hadoop0.17.0). But it failed. the configureations are listed below:
>> for hbase-site.xml:
>>  <configuration>
>>   <property>
>>     <name>hbase.master</name>
>>     <value>machine1:60000</value>
>>     <description>The host and port that the HBase master runs at.
>>     </description>
>>   </property>
>>
>>   <property>
>>     <name>hbase.rootdir</name>
>>     <value>hdfs://machine1:9000/hbase</value>
>>     <description>The directory shared by region servers.
>>     </description>
>>   </property>
>>
>> </configuration>
>>
>> for regionserver file:
>>     machine2
>>     machine3
>>     machine4
>>
>> The hbase are installed on all the four machines and configuration
>> parameters are the same
>>
>>    
Your configuation looks good to me. What issues are you seeing?

Thanks,
St.Ack

>> Any one can help me to clarify the questions?
>>
>> Thanks & Regards
>> Guangfeng
>>
>>
>>
>>
>>
>>    
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: need help for setting muti-node hbase on hadoop

晋光峰
Thanks for your instructions!

Now i have setup the hbase following the configurations on the machines and i can startup hbase after hdfs started. But when I use hql to execute some commands, the error happened. The shell console list many messages below:

Retrying connect to server: /127.0.0.2:60020. Already tried 10 time(s).
08/06/06 10:11:05 INFO ipc.RPC: Server at /127.0.0.2:60020 not available yet, Zzzzz...

such kind of messages are printed out many times and no finish point. The region server is another machine and with different IP address, i wonder why it said that "127.0.0.2:60020 not available ".
When i try to close the hbase, also such kind of messages come out and can't stop it. In one region server, the logs are below:

2008-06-06 10:04:41,082 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=60020, port=60020
2008-06-06 10:04:41,162 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: gf/172.16.0.21:60000. Already tried 1 time(s).
2008-06-06 10:04:43,195 INFO org.apache.hadoop.hbase.HLog: New log writer created at /hbase/log_127.0.0.2_1212717881132_60020/hlog.dat.000
2008-06-06 10:04:43,243 INFO org.mortbay.util.Credential: Checking Resource aliases
2008-06-06 10:04:43,273 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4
2008-06-06 10:04:43,273 INFO org.mortbay.util.Container: Started HttpContext[/static,/static]
2008-06-06 10:04:43,274 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs]
2008-06-06 10:04:43,625 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@68a0864f
2008-06-06 10:04:43,658 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/]
2008-06-06 10:04:43,659 INFO org.mortbay.http.SocketListener: Started SocketListener on 0.0.0.0:60030
2008-06-06 10:04:43,659 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@496614e7
2008-06-06 10:04:43,660 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2008-06-06 10:04:43,661 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 60020: starting
2008-06-06 10:04:43,661 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 60020: starting
2008-06-06 10:04:43,661 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 60020: starting
2008-06-06 10:04:43,661 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 60020: starting
2008-06-06 10:04:43,661 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 60020: starting
2008-06-06 10:04:43,661 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 60020: starting
2008-06-06 10:04:43,661 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020: starting
2008-06-06 10:04:43,662 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 60020: starting
2008-06-06 10:04:43,662 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 60020: starting
2008-06-06 10:04:43,662 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 60020: starting
2008-06-06 10:04:43,662 INFO org.apache.hadoop.hbase.HRegionServer: HRegionServer started at: 127.0.0.2:60020
2008-06-06 10:04:43,662 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 60020: starting
2008-06-06 10:04:43,667 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : -ROOT-,,0
2008-06-06 10:04:43,739 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
2008-06-06 10:04:43,740 INFO org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2008-06-06 10:04:43,747 INFO org.apache.hadoop.hbase.HRegion: region -ROOT-,,0/70236052 available
2008-06-06 10:04:43,747 INFO org.apache.hadoop.hbase.HRegion: checking compaction on region -ROOT-,,0
2008-06-06 10:04:43,749 INFO org.apache.hadoop.hbase.HRegion: checking compaction completed on region -ROOT-,,0 in 0sec
2008-06-06 10:15:06,420 INFO org.apache.hadoop.hbase.HRegionServer: Got quiesce server message
2008-06-06 10:15:06,420 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGIONSERVER_QUIESCE :
2008-06-06 10:16:12,440 WARN org.apache.hadoop.hbase.HRegionServer: Processing message (Retry: 0)
java.net.SocketTimeoutException: timed out waiting for rpc response
        at org.apache.hadoop.ipc.Client.call(Client.java:514)
        at org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:210)
        at $Proxy0.regionServerReport(Unknown Source)
        at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java:737)
        at java.lang.Thread.run(Thread.java:619)
2008-06-06 10:16:12,441 FATAL org.apache.hadoop.hbase.HRegionServer: unable to report to master for 63011 milliseconds - aborting server
2008-06-06 10:16:12,441 INFO org.apache.hadoop.hbase.Leases: regionserver/0:0:0:0:0:0:0:0:60020 closing leases
2008-06-06 10:16:12,441 INFO org.apache.hadoop.hbase.Leases$LeaseMonitor: regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker exiting
2008-06-06 10:16:12,441 INFO org.apache.hadoop.hbase.Leases: regionserver/0:0:0:0:0:0:0:0:60020 closed leases
2008-06-06 10:16:12,441 INFO org.apache.hadoop.ipc.Server: Stopping server on 60020

Any ideas about such kind of error?

Thanks & Regards

2008/6/5, stack <[hidden email]>:

> 晋光峰 wrote:
>>> So my questions are:
>>>     1, Do i need to install hbase software on all the four machines?
>>>    
> Yes.
>
>>>     2, Is the configuration parameters in hbase master node are same as
>>> hbase region server node?
>>>    
> Yes.
>
>>>     3, Is hbase-0.1.2 support lates hadoop-0.17.0?
>>>    
>
> No. HBase runs on Hadoop 0.16.x. The next hbase release, 0.2.0, will
> require Hadoop 0.17.
>
>>> I have tried to install on the four machines(hbase0.1.2 with
>>> hadoop0.17.0). But it failed. the configureations are listed below:
>>> for hbase-site.xml:
>>>  <configuration>
>>>   <property>
>>>     <name>hbase.master</name>
>>>     <value>machine1:60000</value>
>>>     <description>The host and port that the HBase master runs at.
>>>     </description>
>>>   </property>
>>>
>>>   <property>
>>>     <name>hbase.rootdir</name>
>>>     <value>hdfs://machine1:9000/hbase</value>
>>>     <description>The directory shared by region servers.
>>>     </description>
>>>   </property>
>>>
>>> </configuration>
>>>
>>> for regionserver file:
>>>     machine2
>>>     machine3
>>>     machine4
>>>
>>> The hbase are installed on all the four machines and configuration
>>> parameters are the same
>>>
>>>    
> Your configuation looks good to me. What issues are you seeing?
>
> Thanks,
> St.Ack
>
>>> Any one can help me to clarify the questions?
>>>
>>> Thanks & Regards
>>> Guangfeng
>>>
>>>
>>>
>>>
>>>
>>>    
>>
>>  
>
>