[pgpool-general: 6437] Re: High I/O Usage on PGPool nodes

[Tatsuo Ishii]

> Hi,
> 
> Apologies if this has been discussed before, but if so I can’t find a relevant thread.

I think this has not been discussed before. Thank you for the report.

> We are experiencing some strange behavior on our PGPool nodes. In some of them, load average is well in excess of 100 – although CPU is over 70% idle (rest spent in system), so this pointed to I/O. Checking on AWS, we see that there is a lot of disk activity, including writes. This was surprising, as we did not expect the pgpool machines to have any significant disk usage, nor we can find anything in the documentation that would indicate it to be the case.
> 
> After some sysdig usage, the only file getting heavily written to appears to be /var/log/postgresql/pgpool_status
> 
> It is multiple 6 bytes writes a second, all with the same data, by pgpool. So this shouldn’t be a problem as it is not too much data. Except that it seems to be getting flushed right away (fsync?).
> 
> Is this behavior expected? The PG node status have not changed at any time, there are two nodes, always up. Why would the file get rewritten dozens (even hundreds, depending on load) of times per second?
> 
> Would this be caused by a misconfiguration?

I appears that every time a client connects to Pgpool-II, and
Pgpool-II creates new connections to PostgreSQL node, it issues
fsync(). Maybe you disabled connection cache or set very short life
time for connection cache?

Anyway, I think current behavior of Pgpool-II is not best optimized
and I would like to propose attached patch to enhance it. It eliminate
the necessity of fsync(): instead of every time a connection is
created to PostgreSQL, fsync() will be issued only once for the first
time a connection is created to PostgreSQL.

The patch should be able to applied to 3.7.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: too_many_fsync.diff
Type: text/x-patch
Size: 854 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20190227/e31421f2/attachment.bin>