HBase Thrift - HTTP - Kerberos & SPNEGO

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

HBase Thrift - HTTP - Kerberos & SPNEGO

Kevin Risden-2
I'm not 100% sure this should be posted to user list, but starting here
before dev list/JIRA.

I've been working on setting up the Hue HBase and it requires HBase Thrift
v1 server. To support impersonation/proxyuser, the documentation states
that this must be done with HTTP and not binary mode. The cluster has
Kerberos and so the final setup ends up being HBase Thrift in HTTP mode
with Kerberos.

While setting up the HBase Thrift server with HTTP, there were a
significant amount of 401 errors where the HBase Thrift wasn't able to
handle the incoming Kerberos request. Documentation online is sparse when
it comes to setting up the principal/keytab for HTTP Kerberos.

I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
principal/keytab like other Thrift based servers (HiveServer2). It looks
like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
were very close to the same at one point. I made the following changes to
HBase Thrift v1 server implementation to make it work:
* add SPNEGO principal/keytab if in HTTP mode
* return 401 immediately if no authorization header instead of waiting for
try/catch down in program flow

The code changes are available here:
https://github.com/risdenk/hortonworks-hbase-release/compare/HDP-2.5.3.126-base...fix_hbase_thrift_spnego

Does this seem like the right approach?

The same types of changes should apply to master as well. If this looks
reasonable, I can create a JIRA and generate patch against Apache HBase
master.

Side note: I saw the notes about HBase Thrift v1 was meant to go away at
some point but looks like it is still being depended on.

Kevin Risden
Reply | Threaded
Open this post in threaded view
|

Re: HBase Thrift - HTTP - Kerberos & SPNEGO

Josh Elser-2
Hey Kevin!

Looks like you got some good changes in here.

IMO, the HBase Thrift2 "implementation" makes more sense to me (I'm sure
there was a reason for having HTTP be involved at one point, but Thrift
today has the ability to do all of this RPC work for us). I'm not sure
what the HBase API implementations look like between the two versions.

If you'd like to open up a JIRA and throw up a patch, you'd definitely
have my attention if no one else's :)

On 1/11/18 9:31 AM, Kevin Risden wrote:

> I'm not 100% sure this should be posted to user list, but starting here
> before dev list/JIRA.
>
> I've been working on setting up the Hue HBase and it requires HBase Thrift
> v1 server. To support impersonation/proxyuser, the documentation states
> that this must be done with HTTP and not binary mode. The cluster has
> Kerberos and so the final setup ends up being HBase Thrift in HTTP mode
> with Kerberos.
>
> While setting up the HBase Thrift server with HTTP, there were a
> significant amount of 401 errors where the HBase Thrift wasn't able to
> handle the incoming Kerberos request. Documentation online is sparse when
> it comes to setting up the principal/keytab for HTTP Kerberos.
>
> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
> principal/keytab like other Thrift based servers (HiveServer2). It looks
> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
> were very close to the same at one point. I made the following changes to
> HBase Thrift v1 server implementation to make it work:
> * add SPNEGO principal/keytab if in HTTP mode
> * return 401 immediately if no authorization header instead of waiting for
> try/catch down in program flow
>
> The code changes are available here:
> https://github.com/risdenk/hortonworks-hbase-release/compare/HDP-2.5.3.126-base...fix_hbase_thrift_spnego
>
> Does this seem like the right approach?
>
> The same types of changes should apply to master as well. If this looks
> reasonable, I can create a JIRA and generate patch against Apache HBase
> master.
>
> Side note: I saw the notes about HBase Thrift v1 was meant to go away at
> some point but looks like it is still being depended on.
>
> Kevin Risden
>
Reply | Threaded
Open this post in threaded view
|

Re: HBase Thrift - HTTP - Kerberos & SPNEGO

Kevin Risden-2
"HBase Thrift2 "implementation" makes more sense to me"

I agree with that statement since thrift2 follows the HBase API more
closely. Thrift 1 follows the old HBase API. I don't think using Thrift2
with Hue is an option right now. It still interacts with Thrift 1. (I'm not
really looking to rewrite the Hue HBase Thrift module) There didn't look to
be much code shared between Thrift 1 and Thrift 2 server implementations.
Thrift 1 looks very much like HiveServer2 and the 401 bail out early might
also apply there.

I'll open a JIRA and throw up a patch hopefully this week.

Kevin Risden

On Thu, Jan 11, 2018 at 9:50 AM, Josh Elser <[hidden email]> wrote:

> Hey Kevin!
>
> Looks like you got some good changes in here.
>
> IMO, the HBase Thrift2 "implementation" makes more sense to me (I'm sure
> there was a reason for having HTTP be involved at one point, but Thrift
> today has the ability to do all of this RPC work for us). I'm not sure what
> the HBase API implementations look like between the two versions.
>
> If you'd like to open up a JIRA and throw up a patch, you'd definitely
> have my attention if no one else's :)
>
>
> On 1/11/18 9:31 AM, Kevin Risden wrote:
>
>> I'm not 100% sure this should be posted to user list, but starting here
>> before dev list/JIRA.
>>
>> I've been working on setting up the Hue HBase and it requires HBase Thrift
>> v1 server. To support impersonation/proxyuser, the documentation states
>> that this must be done with HTTP and not binary mode. The cluster has
>> Kerberos and so the final setup ends up being HBase Thrift in HTTP mode
>> with Kerberos.
>>
>> While setting up the HBase Thrift server with HTTP, there were a
>> significant amount of 401 errors where the HBase Thrift wasn't able to
>> handle the incoming Kerberos request. Documentation online is sparse when
>> it comes to setting up the principal/keytab for HTTP Kerberos.
>>
>> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
>> principal/keytab like other Thrift based servers (HiveServer2). It looks
>> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
>> were very close to the same at one point. I made the following changes to
>> HBase Thrift v1 server implementation to make it work:
>> * add SPNEGO principal/keytab if in HTTP mode
>> * return 401 immediately if no authorization header instead of waiting for
>> try/catch down in program flow
>>
>> The code changes are available here:
>> https://github.com/risdenk/hortonworks-hbase-release/compare
>> /HDP-2.5.3.126-base...fix_hbase_thrift_spnego
>>
>> Does this seem like the right approach?
>>
>> The same types of changes should apply to master as well. If this looks
>> reasonable, I can create a JIRA and generate patch against Apache HBase
>> master.
>>
>> Side note: I saw the notes about HBase Thrift v1 was meant to go away at
>> some point but looks like it is still being depended on.
>>
>> Kevin Risden
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: HBase Thrift - HTTP - Kerberos & SPNEGO

Kevin Risden-2
"hopefully this week"... famous last words.

Finally got around to creating a JIRA: HBASE-19852 Close to having the
patch to submit done as well.

Kevin Risden

On Thu, Jan 11, 2018 at 10:02 AM, Kevin Risden <[hidden email]> wrote:

> "HBase Thrift2 "implementation" makes more sense to me"
>
> I agree with that statement since thrift2 follows the HBase API more
> closely. Thrift 1 follows the old HBase API. I don't think using Thrift2
> with Hue is an option right now. It still interacts with Thrift 1. (I'm not
> really looking to rewrite the Hue HBase Thrift module) There didn't look to
> be much code shared between Thrift 1 and Thrift 2 server implementations.
> Thrift 1 looks very much like HiveServer2 and the 401 bail out early might
> also apply there.
>
> I'll open a JIRA and throw up a patch hopefully this week.
>
> Kevin Risden
>
> On Thu, Jan 11, 2018 at 9:50 AM, Josh Elser <[hidden email]> wrote:
>
>> Hey Kevin!
>>
>> Looks like you got some good changes in here.
>>
>> IMO, the HBase Thrift2 "implementation" makes more sense to me (I'm sure
>> there was a reason for having HTTP be involved at one point, but Thrift
>> today has the ability to do all of this RPC work for us). I'm not sure what
>> the HBase API implementations look like between the two versions.
>>
>> If you'd like to open up a JIRA and throw up a patch, you'd definitely
>> have my attention if no one else's :)
>>
>>
>> On 1/11/18 9:31 AM, Kevin Risden wrote:
>>
>>> I'm not 100% sure this should be posted to user list, but starting here
>>> before dev list/JIRA.
>>>
>>> I've been working on setting up the Hue HBase and it requires HBase
>>> Thrift
>>> v1 server. To support impersonation/proxyuser, the documentation states
>>> that this must be done with HTTP and not binary mode. The cluster has
>>> Kerberos and so the final setup ends up being HBase Thrift in HTTP mode
>>> with Kerberos.
>>>
>>> While setting up the HBase Thrift server with HTTP, there were a
>>> significant amount of 401 errors where the HBase Thrift wasn't able to
>>> handle the incoming Kerberos request. Documentation online is sparse when
>>> it comes to setting up the principal/keytab for HTTP Kerberos.
>>>
>>> I noticed that the HBase Thrift HTTP implementation was missing SPNEGO
>>> principal/keytab like other Thrift based servers (HiveServer2). It looks
>>> like HiveServer2 Thrift implementation and HBase Thrift v1 implementation
>>> were very close to the same at one point. I made the following changes to
>>> HBase Thrift v1 server implementation to make it work:
>>> * add SPNEGO principal/keytab if in HTTP mode
>>> * return 401 immediately if no authorization header instead of waiting
>>> for
>>> try/catch down in program flow
>>>
>>> The code changes are available here:
>>> https://github.com/risdenk/hortonworks-hbase-release/compare
>>> /HDP-2.5.3.126-base...fix_hbase_thrift_spnego
>>>
>>> Does this seem like the right approach?
>>>
>>> The same types of changes should apply to master as well. If this looks
>>> reasonable, I can create a JIRA and generate patch against Apache HBase
>>> master.
>>>
>>> Side note: I saw the notes about HBase Thrift v1 was meant to go away at
>>> some point but looks like it is still being depended on.
>>>
>>> Kevin Risden
>>>
>>>
>