Coprocessor prePut

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Coprocessor prePut

Yang Zhang
Hello Everyone

        I am using the coprocessor and want to put another data when
someone put a data into table. So I try prePut(), but when you call
HRegion.put(), this will call my coprocessor's prePut function again, and
it will cause an dead loop.

        My code looks like below, If anyone know how to make HBase  call my
coprocessor just for the first time?

@Override
    public void prePut(ObserverContext<RegionCoprocessorEnvironment>
context, Put put, WALEdit edit,
                       Durability durability) throws IOException {
        logger.info("==== Coprocessor shouldComplete :
"+context.shouldComplete()+"====");
        context.bypass();

        Put myPut = new Put(new RowKey(1).toBytes());

        //this call will caused an dead loop
        context.getEnvironment().getRegion().put(put);

    }

Thanks anyway
Reply | Threaded
Open this post in threaded view
|

Re: Coprocessor prePut

Ted Yu-3
Can you clarify your use case ?

bq. put a data into table

Does your coprocessor write to the same table which receives user data ?

Cheers

On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <[hidden email]> wrote:

> Hello Everyone
>
>         I am using the coprocessor and want to put another data when
> someone put a data into table. So I try prePut(), but when you call
> HRegion.put(), this will call my coprocessor's prePut function again, and
> it will cause an dead loop.
>
>         My code looks like below, If anyone know how to make HBase  call my
> coprocessor just for the first time?
>
> @Override
>     public void prePut(ObserverContext<RegionCoprocessorEnvironment>
> context, Put put, WALEdit edit,
>                        Durability durability) throws IOException {
>         logger.info("==== Coprocessor shouldComplete :
> "+context.shouldComplete()+"====");
>         context.bypass();
>
>         Put myPut = new Put(new RowKey(1).toBytes());
>
>         //this call will caused an dead loop
>         context.getEnvironment().getRegion().put(put);
>
>     }
>
> Thanks anyway
>
Reply | Threaded
Open this post in threaded view
|

Re: Coprocessor prePut

Yang Zhang
Yes, It is the same table.

2018-01-23 1:46 GMT+08:00 Ted Yu <[hidden email]>:

> Can you clarify your use case ?
>
> bq. put a data into table
>
> Does your coprocessor write to the same table which receives user data ?
>
> Cheers
>
> On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <[hidden email]>
> wrote:
>
> > Hello Everyone
> >
> >         I am using the coprocessor and want to put another data when
> > someone put a data into table. So I try prePut(), but when you call
> > HRegion.put(), this will call my coprocessor's prePut function again, and
> > it will cause an dead loop.
> >
> >         My code looks like below, If anyone know how to make HBase  call
> my
> > coprocessor just for the first time?
> >
> > @Override
> >     public void prePut(ObserverContext<RegionCoprocessorEnvironment>
> > context, Put put, WALEdit edit,
> >                        Durability durability) throws IOException {
> >         logger.info("==== Coprocessor shouldComplete :
> > "+context.shouldComplete()+"====");
> >         context.bypass();
> >
> >         Put myPut = new Put(new RowKey(1).toBytes());
> >
> >         //this call will caused an dead loop
> >         context.getEnvironment().getRegion().put(put);
> >
> >     }
> >
> > Thanks anyway
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Coprocessor prePut

Ted Yu-3
Your prePut would write to a different column in the table, right ?
Otherwise do you prevent normal writes from getting into the same column(s)
?

If your prePut writes to dedicated column(s), you can check the presence of
such column(s) so that the call is not infinite.

Cheers

On Mon, Jan 22, 2018 at 8:56 PM, Yang Zhang <[hidden email]> wrote:

> Yes, It is the same table.
>
> 2018-01-23 1:46 GMT+08:00 Ted Yu <[hidden email]>:
>
> > Can you clarify your use case ?
> >
> > bq. put a data into table
> >
> > Does your coprocessor write to the same table which receives user data ?
> >
> > Cheers
> >
> > On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <[hidden email]>
> > wrote:
> >
> > > Hello Everyone
> > >
> > >         I am using the coprocessor and want to put another data when
> > > someone put a data into table. So I try prePut(), but when you call
> > > HRegion.put(), this will call my coprocessor's prePut function again,
> and
> > > it will cause an dead loop.
> > >
> > >         My code looks like below, If anyone know how to make HBase
> call
> > my
> > > coprocessor just for the first time?
> > >
> > > @Override
> > >     public void prePut(ObserverContext<RegionCoprocessorEnvironment>
> > > context, Put put, WALEdit edit,
> > >                        Durability durability) throws IOException {
> > >         logger.info("==== Coprocessor shouldComplete :
> > > "+context.shouldComplete()+"====");
> > >         context.bypass();
> > >
> > >         Put myPut = new Put(new RowKey(1).toBytes());
> > >
> > >         //this call will caused an dead loop
> > >         context.getEnvironment().getRegion().put(put);
> > >
> > >     }
> > >
> > > Thanks anyway
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Coprocessor prePut

Yang Zhang
In fact, I prevent normal writes, and put the same data with a different
row key(Except row key, they have the same columns and same data).
I have already solved the infinite call by add and check some flag
attribute to Put. If the flag exist then skip my code.

This will prevent the infinite call of prePut(). But I want to know If
there are any method support by coprocessor context to do that.
Just like that you can call context.complete() to skip other coprocessors.

Thanks for your advice

2018-01-23 13:01 GMT+08:00 Ted Yu <[hidden email]>:

> Your prePut would write to a different column in the table, right ?
> Otherwise do you prevent normal writes from getting into the same column(s)
> ?
>
> If your prePut writes to dedicated column(s), you can check the presence of
> such column(s) so that the call is not infinite.
>
> Cheers
>
> On Mon, Jan 22, 2018 at 8:56 PM, Yang Zhang <[hidden email]>
> wrote:
>
> > Yes, It is the same table.
> >
> > 2018-01-23 1:46 GMT+08:00 Ted Yu <[hidden email]>:
> >
> > > Can you clarify your use case ?
> > >
> > > bq. put a data into table
> > >
> > > Does your coprocessor write to the same table which receives user data
> ?
> > >
> > > Cheers
> > >
> > > On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <[hidden email]>
> > > wrote:
> > >
> > > > Hello Everyone
> > > >
> > > >         I am using the coprocessor and want to put another data when
> > > > someone put a data into table. So I try prePut(), but when you call
> > > > HRegion.put(), this will call my coprocessor's prePut function again,
> > and
> > > > it will cause an dead loop.
> > > >
> > > >         My code looks like below, If anyone know how to make HBase
> > call
> > > my
> > > > coprocessor just for the first time?
> > > >
> > > > @Override
> > > >     public void prePut(ObserverContext<RegionCoprocessorEnvironment>
> > > > context, Put put, WALEdit edit,
> > > >                        Durability durability) throws IOException {
> > > >         logger.info("==== Coprocessor shouldComplete :
> > > > "+context.shouldComplete()+"====");
> > > >         context.bypass();
> > > >
> > > >         Put myPut = new Put(new RowKey(1).toBytes());
> > > >
> > > >         //this call will caused an dead loop
> > > >         context.getEnvironment().getRegion().put(put);
> > > >
> > > >     }
> > > >
> > > > Thanks anyway
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Coprocessor prePut

Ted Yu-3
bq. can call context.complete() to skip other coprocessors.

FYI in hbase-2, complete() has been folded into bypass().
See javadoc of bypass() for details.

On Mon, Jan 22, 2018 at 9:34 PM, Yang Zhang <[hidden email]> wrote:

> In fact, I prevent normal writes, and put the same data with a different
> row key(Except row key, they have the same columns and same data).
> I have already solved the infinite call by add and check some flag
> attribute to Put. If the flag exist then skip my code.
>
> This will prevent the infinite call of prePut(). But I want to know If
> there are any method support by coprocessor context to do that.
> Just like that you can call context.complete() to skip other coprocessors.
>
> Thanks for your advice
>
> 2018-01-23 13:01 GMT+08:00 Ted Yu <[hidden email]>:
>
> > Your prePut would write to a different column in the table, right ?
> > Otherwise do you prevent normal writes from getting into the same
> column(s)
> > ?
> >
> > If your prePut writes to dedicated column(s), you can check the presence
> of
> > such column(s) so that the call is not infinite.
> >
> > Cheers
> >
> > On Mon, Jan 22, 2018 at 8:56 PM, Yang Zhang <[hidden email]>
> > wrote:
> >
> > > Yes, It is the same table.
> > >
> > > 2018-01-23 1:46 GMT+08:00 Ted Yu <[hidden email]>:
> > >
> > > > Can you clarify your use case ?
> > > >
> > > > bq. put a data into table
> > > >
> > > > Does your coprocessor write to the same table which receives user
> data
> > ?
> > > >
> > > > Cheers
> > > >
> > > > On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Hello Everyone
> > > > >
> > > > >         I am using the coprocessor and want to put another data
> when
> > > > > someone put a data into table. So I try prePut(), but when you call
> > > > > HRegion.put(), this will call my coprocessor's prePut function
> again,
> > > and
> > > > > it will cause an dead loop.
> > > > >
> > > > >         My code looks like below, If anyone know how to make HBase
> > > call
> > > > my
> > > > > coprocessor just for the first time?
> > > > >
> > > > > @Override
> > > > >     public void prePut(ObserverContext<
> RegionCoprocessorEnvironment>
> > > > > context, Put put, WALEdit edit,
> > > > >                        Durability durability) throws IOException {
> > > > >         logger.info("==== Coprocessor shouldComplete :
> > > > > "+context.shouldComplete()+"====");
> > > > >         context.bypass();
> > > > >
> > > > >         Put myPut = new Put(new RowKey(1).toBytes());
> > > > >
> > > > >         //this call will caused an dead loop
> > > > >         context.getEnvironment().getRegion().put(put);
> > > > >
> > > > >     }
> > > > >
> > > > > Thanks anyway
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Coprocessor prePut

Anoop John
What fix u tried now will help you.  You can not avoid the loop by
using the complete or bypass way.. Because that is on the present
context.  Another put on region will create a new context and so that
continues.   One more thing/suggestion would be to see the hook
preBatchMutate.  You will receive the Mutations being processed in a
MiniBatchOperationInProgress object to which u can add more Mutations
(Puts) as CP added.  These also will get executed.  As you are writing
to the same Region, this will work.

-Anoop-

On Tue, Jan 23, 2018 at 11:25 AM, Ted Yu <[hidden email]> wrote:

> bq. can call context.complete() to skip other coprocessors.
>
> FYI in hbase-2, complete() has been folded into bypass().
> See javadoc of bypass() for details.
>
> On Mon, Jan 22, 2018 at 9:34 PM, Yang Zhang <[hidden email]> wrote:
>
>> In fact, I prevent normal writes, and put the same data with a different
>> row key(Except row key, they have the same columns and same data).
>> I have already solved the infinite call by add and check some flag
>> attribute to Put. If the flag exist then skip my code.
>>
>> This will prevent the infinite call of prePut(). But I want to know If
>> there are any method support by coprocessor context to do that.
>> Just like that you can call context.complete() to skip other coprocessors.
>>
>> Thanks for your advice
>>
>> 2018-01-23 13:01 GMT+08:00 Ted Yu <[hidden email]>:
>>
>> > Your prePut would write to a different column in the table, right ?
>> > Otherwise do you prevent normal writes from getting into the same
>> column(s)
>> > ?
>> >
>> > If your prePut writes to dedicated column(s), you can check the presence
>> of
>> > such column(s) so that the call is not infinite.
>> >
>> > Cheers
>> >
>> > On Mon, Jan 22, 2018 at 8:56 PM, Yang Zhang <[hidden email]>
>> > wrote:
>> >
>> > > Yes, It is the same table.
>> > >
>> > > 2018-01-23 1:46 GMT+08:00 Ted Yu <[hidden email]>:
>> > >
>> > > > Can you clarify your use case ?
>> > > >
>> > > > bq. put a data into table
>> > > >
>> > > > Does your coprocessor write to the same table which receives user
>> data
>> > ?
>> > > >
>> > > > Cheers
>> > > >
>> > > > On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <[hidden email]
>> >
>> > > > wrote:
>> > > >
>> > > > > Hello Everyone
>> > > > >
>> > > > >         I am using the coprocessor and want to put another data
>> when
>> > > > > someone put a data into table. So I try prePut(), but when you call
>> > > > > HRegion.put(), this will call my coprocessor's prePut function
>> again,
>> > > and
>> > > > > it will cause an dead loop.
>> > > > >
>> > > > >         My code looks like below, If anyone know how to make HBase
>> > > call
>> > > > my
>> > > > > coprocessor just for the first time?
>> > > > >
>> > > > > @Override
>> > > > >     public void prePut(ObserverContext<
>> RegionCoprocessorEnvironment>
>> > > > > context, Put put, WALEdit edit,
>> > > > >                        Durability durability) throws IOException {
>> > > > >         logger.info("==== Coprocessor shouldComplete :
>> > > > > "+context.shouldComplete()+"====");
>> > > > >         context.bypass();
>> > > > >
>> > > > >         Put myPut = new Put(new RowKey(1).toBytes());
>> > > > >
>> > > > >         //this call will caused an dead loop
>> > > > >         context.getEnvironment().getRegion().put(put);
>> > > > >
>> > > > >     }
>> > > > >
>> > > > > Thanks anyway
>> > > > >
>> > > >
>> > >
>> >
>>