[Pgpool-general] unexpected EOF on client connection

[Tatsuo Ishii]

> On Wed, Sep 14, 2011 at 4:22 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
>>>>> I'm pretty sure that's not the case as the messages stop whenever
>>>>> pgpool isn't running, they were not present prior to using pgpool, and
>>>>> pg_hba.conf is setup such that the database servers only accept
>>>>> connections from each other, and the server running pgpool.  None of
>>>>> these servers have normal users connected directly to them (such as
>>>>> with ssh), nor are they running anything that would connect to the
>>>>> database as a client.  Also, the volume of these messages are such
>>>>> that something significant has to be causing them.  Last night, in the
>>>>> span of 5 minutes, there were 117 of these messages.
>>>>
>>>> Ok. I would like to narraow down the reason why we have "unexpected
>>>> EOF on client connection" message frequently. I think currently there
>>>> are two possiblities:
>>>>
>>>> 1) pgpool child was killed by some unknown reason(we can omit
>>>>   segfault case because you don't see it in the pgpool log)
>>>>
>>>> 2) pgpool child disconnects to PostgreSQL in ungraceful manner
>>>>
>>>> For 1) I would like to know if pgpool child process are fine since
>>>> they are spawned. Are you seeing any pgpool child process disappeared
>>>> since pgpool started?
>>>
>>> I assume this should be determined by num_init_children (which I've
>>> set to 195 in pgpool.conf)?  If so, then I currently have 195
>>> processes in either the "wait for connection request" state or
>>> actively connected state.
>>
>> No. Pgpool parent process automatically respawns child process if it's
>> dyning. So having num_init_children child process is not showing
>> anything usefull. You record 195 process ids and compare current
>> process ids. If some of them have been changed, we can assume that
>> child process is dying.
> 
> Ah, good point.  I just diffed the list of PIDs associated with pgpool
> processes before and after another EOF message in the log, and there
> were no differences.  So I think that rules out any processes dying?

Right.

> One other thing that I just noticed from comparing logs between all of
> the database servers is that the time stamps for every one of the
> 'unexpected EOF on client connection' instances are identical.  In
> other words, they are happening at the same time on each server.  I
> think this further suggests that pgpool has to be doing it?

Yes, I think so unless you set connection_life_time to other than 0 or
the network connection between PostgreSQL and pgpool is unstable.

Let me think how we can make further investigation...
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp