Quantcast

Help with Selective replication in Hbase

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Help with Selective replication in Hbase

techbuddy
Hi,

I need to be able to selectively replicate the mutations from my master cluster into my backup/slave cluster,based on some application specific policy.In other words I want to suppress replication of certain mutation events, and allow the same only for the rest.

The ReplicationSinkService interface (http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/ReplicationSinkService.html) looks promising in this regard, as it provides the application a hook into the replication process.

I intend to extend the Replication class that implements this interface and override the replicateLogEntries(HLog.Entry[] entries) method. However I couldn't find any config parameter (that I can specify in hbase-site.xml) to specify my custom Replication class.

Any pointers on this regard would be greatly appreciated.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with Selective replication in Hbase

techbuddy
Looks like I got what I was looking for. Will try this out.

 String sinkClassname = conf.get(HConstants.REPLICATION_SINK_SERVICE_CLASSNAME,
3549                             HConstants.REPLICATION_SERVICE_CLASSNAME_DEFAULT);
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with Selective replication in Hbase

techbuddy

Since the behavior of this application controlled replication (Through ReplicationSourceService and ReplicationSinkService) is not documented ,we were wondering if this feature runs the possibility of being withdrawn in future.

Also, has anybody used this in production/prototyping? If so, please share your experience.

-thanks
SB
Loading...