[pgpool-general: 6943] Re: Keberos issues

[Oliver Freyermuth]

Dear Bo,

thanks for your reply!

Am 27.03.20 um 07:24 schrieb Bo Peng:
> Hello,
> 
> On Fri, 20 Mar 2020 09:57:23 +0100
> Oliver Freyermuth <freyermuth at physik.uni-bonn.de> wrote:
> 
>> Dear PGPoolers,
>>
>> after setting up PGPool[0] with md5 auth and having the pool_hba correctly set up,
>> and login working fine, I encountered one client which could not log in with the following message
>> (just using the CLI psql tool):
>> --------------------
>> $ psql -l "host=my_pgpool_node port=5432 dbname=testdb user=testuser"
>> psql: error: could not connect to server: server closed the connection unexpectedly
>>           This probably means the server terminated abnormally
>>           before or while processing the request.
>> --------------------
>> So the client does not see any underlying cause.
>>
>> Checking with strace, I found the client seems to try Kerberos 5 first, and apparently never falls back to password auth
>> (it does not prompt for a password).
>>
>> Checking the pgpool-II debug logs:
>> --------------------
>> Mar 20 09:46:34 pgsql pgpool[25144]: [949-1] 2020-03-20 09:46:34: pid 25144: LOG:  new connection received
>> Mar 20 09:46:34 pgsql pgpool[25144]: [949-2] 2020-03-20 09:46:34: pid 25144: DETAIL:  connecting host=client_ip port=54946
>> Mar 20 09:46:34 pgsql pgpool[25144]: [949-3] 2020-03-20 09:46:34: pid 25144: LOCATION:  child.c:2166
>> Mar 20 09:46:34 pgsql pgpool[25144]: [951-1] 2020-03-20 09:46:34: pid 25144: FATAL:  client authentication failed
>> Mar 20 09:46:34 pgsql pgpool[25144]: [951-2] 2020-03-20 09:46:34: pid 25144: DETAIL:  no pool_hba.conf entry for host "client_ip", user "", database "", SSL off
>> Mar 20 09:46:34 pgsql pgpool[25144]: [951-3] 2020-03-20 09:46:34: pid 25144: HINT:  see pgpool log for details
>> --------------------
>> it's quite curious that "user" and "database" are empty.
>>
>> Reading through the code, notably check_hba() and ClientAuthentication() in src/auth/pool_hba.c,
>> my best guess is that a client connecting with Kerberos is implicitly rejected and not asked to retry with another method.
>> Sadly, there seems to be no way to disallow psql from trying KRB5 (and it prefers it if possible).
>>
>> So the only approach I've found for the moment is to destroy KRB credentials to force psql to use MD5 auth,
>> that does indeed work, but any client who is capable of KRB5 auth will silently fail to authenticate with MD5 against PGPool
>> since the connection is dropped before it can try.
>>
>> I'd interpret this as a bug, but before opening an issue, wanted to ask here if this problem is known
>> and / or my understanding is correct.
>>
>> Cheers,
>> 	Oliver
>>
>> [0] Version: pgpool-II-pg11-4.0.8-1.pgdg.rhel7 (on CentOS 7)
>>
> 
> I think it is a existing issue on CentOS6.
> 
> [FAQ]
> https://www.pgpool.net/mediawiki/index.php/FAQ#Connection_failed_in_CentOS6
> 
> I tried on CentOS7 but I couldn't reproduce this issue.
> Could you share your pgpool.conf, pool_hba.conf and pg_hba.conf?
>

It seems I forgot to mention what was used on the client. The client is using the PostgreSQL 12 client tools, compiled with GSSAPI / Kerberos 5 support
(it's a Gentoo Linux system).

I performed the following tests now:
- Trying "export PGGSSENCMODE=disable" on that client.
   Indeed, that worked around the problem!
- Trying to reproduce with a CentOS 7 client. I couldn't.
- Trying to reproduce with a Ubuntu 18.04 client. I couldn't.

Since this means the server configuration is not the cause, I did not (yet) include the config files, I can still provide them if you want.

So I agree, this should be the FAQ entry you linked. Since Gentoo uses a rather "vanilla" PostgreSQL build, I checked which patches Debian / Ubuntu and CentOS 7 apply,
but could not find anything related.
So the cause likely is that old CentOS 6 and rather modern Gentoo both try GSSAPI / Kerberos 5 first. I wonder if we can pinpoint the actual cause of that —
the only difference I see right now is that Gentoo is already at PostgreSQL 12, and already ships the 1.17 Kerberos libraries.

Are you aware of any other components involved?
I can certainly pinpoint this if I have an idea of where to start looking. Another solution would probably be if pgpool denied GSSAPI and caused the client to rather use password auth
(if that is possible?).

Cheers,
	Oliver

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5432 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20200327/7a31cdac/attachment.p7s>