[pgpool-general: 2178] Re: read_startup_packet: incorrect packet length What does that mean?

Lachezar Dobrev l.dobrev at gmail.com
Wed Oct 9 20:29:50 JST 2013


  Ahh... Had similar problems with Nagios: my administrator had set
only port-knocking to check if the database is alive and running. I
had to make him replace the port-knocking with an actual PostgreSQL
specific check (check_pgsql). A quick check on the Zabbix page shows
that it *should* have support for monitoring PostgreSQL, although the
actual implementation eludes me (it might be a simple port-knock).

2013/10/9 Mistina Michal <Michal.Mistina at virte.sk>:
> Hi Tatsuo.
> Setting of the log_connections setting helped.
>
> The log showed this:
> 2013-10-09 08:35:01 ERROR: pid 28315: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:35:17 LOG:   pid 31625: connection received: host=127.0.0.1
> port=59225
> 2013-10-09 08:35:17 ERROR: pid 31625: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:35:31 LOG:   pid 28640: connection received: host=127.0.0.1
> port=59229
> 2013-10-09 08:35:31 LOG:   pid 6187: do_child: failback event found. restart
> myself.
> 2013-10-09 08:35:31 ERROR: pid 28640: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:35:47 LOG:   pid 29594: connection received: host=127.0.0.1
> port=59235
> 2013-10-09 08:35:47 ERROR: pid 29594: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:36:01 LOG:   pid 28800: connection received: host=127.0.0.1
> port=59239
> 2013-10-09 08:36:01 ERROR: pid 28800: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:36:17 LOG:   pid 6200: do_child: failback event found. restart
> myself.
> 2013-10-09 08:36:17 LOG:   pid 29099: connection received: host=127.0.0.1
> port=59247
> 2013-10-09 08:36:17 ERROR: pid 29099: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:36:31 LOG:   pid 32454: connection received: host=127.0.0.1
> port=59252
> 2013-10-09 08:36:31 ERROR: pid 32454: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:36:47 LOG:   pid 29429: connection received: host=127.0.0.1
> port=59258
> 2013-10-09 08:36:47 ERROR: pid 29429: read_startup_packet: incorrect packet
> length (-1928938790)
> 2013-10-09 08:37:01 LOG:   pid 28640: connection received: host=127.0.0.1
> port=59262
> 2013-10-09 08:37:01 ERROR: pid 28640: read_startup_packet: incorrect packet
> length (-1928938790)
>
> As I found out from my colleague the zabbix agent checked through its
> trigger if the Pgpool and PostgreSQL port is alive. If he stopped it, the
> messages went away.
>
> Hopefully I helped somebody.
>
> Best regards,
> Michal Mistina
> -----Original Message-----
> From: Tatsuo Ishii [mailto:ishii at postgresql.org]
> Sent: Tuesday, October 8, 2013 8:15 AM
> To: Mistina Michal
> Cc: pgpool-general at pgpool.net
> Subject: Re: [pgpool-general: 2141] Re: read_startup_packet: incorrect
> packet length What does that mean?
>
>> Hi Tatsuo,
>>> Hi Michal,
>>>> Hi Tatsuo.
>>>> Sorry for the delayed answer.
>>>> In the meantime I've changed the configuration, but the message
>>>> "read_startup_packet: incorrect packet length" didn't go away. I am
>>>> attaching configuration at the end of the e-mail and also a log file
>>>> right after I started Pgpool.
>>>>
>>>> I think I understand the issue I've described in the last e-mail. I
>>>> mean the psql -c "show pool_nodes;" -d postgres -U postgres -p 9898
>>>> -> the port should be 9999. Just silly typo.
>>>>
>>>> I would like to summary of the issues I am experiencing now:
>>>> 1. There are periodic ERROR messages in the Pgpool log file -
>>>> "read_startup_packet: incorrect packet length (-1656006549)".
>>>> 2. There are periodic messages in the PostgreSQL log file -
>>>> "incomplete startup packet".
>>>> 3. If I halt or reboot primary node (soft shutdown), the secondary
>>>> node triggers fail-over immediatelly. --> NOT DESIRED BEHAVIOUR 4.
>>>> If I power off the primary node (unexpectedly), the secondary node
>>>> applies health_check and triggers fail-over after health_check
>>>> time-out. --> DESIRED BEHAVIOUR
>>>>
>>>> I think, the first two issues are related. But don't know what could
>> cause it.
>>>> We are using special services which connects to the Pgpool by using
>>>> JDBC. Can the services be the source of these error messages?
>>
>>>No idea. However, you can enable "log_connections" which logs who is
>> connecting to pgpool. This will reveal who is the client when the
>> errors raise.
>> Good point. I will try to investigate it and also asks our developers
>> what could do that. From what I know the services are doing the
>> "SELECT 1" query to determine if the backend is alive. But that is a
> normal thing. We'll see.
>>
>>>>
>>>> According to issue number 3.. I can see in the log file, what happened:
>>>> "postmaster on DB node 0 was shutdown by administrative command".
>>>> How can I force Pgpool to not fail-over after the administrative
>>>> command was issued? I would like to apply same health_check timouts
>>>> also in the situation when the administrative command was issued. It
>>>> is because in our environment one node is represented by more hosts
>>>> where the Postgresql services are migrating from one host to another
>> automatically. It occurs also if the "halt"
>>>> or "reboot" command is issued on that one node. This is not
>>>> errorneous condition for our environment, because if the "halt" is
>>>> issued on one host, PostgreSQL service will be brought up on another
>>>> node. Virtual IP address is in front of it.
>>
>>>I understand your situation. What you can do is setting:
>>
>>>backend_flag0 = 'DISALLOW_TO_FAILOVER'
>>
>>>This will prevent your primary node from fail-over by "shutdown by
>> administrative command". Please note that this will also prevent the
>> primary from fail-over by the health check.
>> This is what I was hoping to avoid. Because it will absolutly disallow
>> to fail-over from primary node to the secondary. But thank you anyway.
>> I am now certain that, there is no other way and I should include
>> additional steps to maintenance manual, which should be taken in the
>> process of restarting one host by administrative command where pgpool
> currently runs.
>>
>> One more question.. do you plan to add the setting of "not to
>> fail-over by administrative command" in the future releases or it is
>> absolutely wrong idea?
>
> IMO, that would be a good idea. I will add it to the TODO list.
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
>
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general
>


More information about the pgpool-general mailing list