[pgpool-general: 6044] Re: wrong arguments for online recovery script
Benjamin Firl
bf at wisit.com
Tue Apr 17 00:48:57 JST 2018
Hi,
is there any news on this topic? Did my example clear things up?
Kind regards
Benjamin Firl
On 04/05/2018 12:23 PM, Benjamin Firl wrote:
> I'll try to explain it with an example from our system:
>
> *
> */~$ psql -U postgres -p 5433 -c "show pool_nodes" | grep -v unused//
> // node_id | hostname | port | status | lb_weight | role |
> select_cnt | load_balance_node | replication_delay//
> //---------+-----------+------+--------+-----------+--------+------------+-------------------+-------------------//
> // 14 | 10.0.1.14 | 5432 | down | 0.200000 | slave |
> 2035 | false | 0//
> // 15 | 10.0.1.15 | 5432 | down | 0.200000 | slave |
> 2181 | false | 0//
> // 19 | 10.0.1.19 | 5432 | down | 0.200000 | slave |
> 1834 | false | 0//
> // 20 | 10.0.1.20 | 5432 | up | 0.200000 | master |
> 3997772 | true | 0//
> // 21 | 10.0.1.21 | 5432 | up | 0.200000 | slave |
> 3781854 | false | 0/
>
>
> We create and delete nodes automatically depending on our current
> resource needs. As you can see, we generate the node ID from the IP
> address of the system.
> Node IDs 14, 15 and 19 were used before, but are not even in the
> configuration file anymore. Node 20 and 21 are the only ones active.
> If we now add a new node, we write it to the configuration file with
> its generated node ID and start the online recovery process. If this
> ID is e.g. 22, everything works fine. But if the ID is e.g. 1 in this
> case, the recovery scripts receives the wrong parameters as shown.
>
> Hope that makes things clearer.
>
>
> Kind regards
> Benjamin Firl
>
>
>
>
> On 04/05/2018 11:01 AM, Tatsuo Ishii wrote:
>> Can you share the exact step to reproduce the problem?
>>
>> "a new lowest node id is added" is a little bit unclear to me.
>>
>> Best regards,
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese:http://www.sraoss.co.jp
>>
>>> Hallo,
>>>
>>>
>>> I tested with the patched 3.7.2, still the same problem. I also tested
>>> again with 3.6.7, this version also shows the same behaviour.
>>>
>>> Also I am now able to reproduce it relatively easy, it really happens
>>> every time a new lowest node id is added. So if there is anything else I
>>> can try, configuration options or something, I will gladly do it.
>>>
>>>
>>> Kind regards
>>> Benjamin Firl
>>>
>>>
>>>
>>> On 03/29/2018 10:21 AM, Tatsuo Ishii wrote:
>>>> Still I suspect the bug is only with 3.7.2. Recently we releasd a
>>>> fixed version of 3.7.2 RPM (actually we just reverted a commit which
>>>> caused the bug). If you like, you could try the RPM.
>>>>
>>>> Or you install from the source code, you could patch 3.7.2 with this commit:
>>>> https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=e8baa3f0ee1a24f833cfa6cc5d0104b5cefe7b04
>>>>
>>>> Best regards,
>>>> --
>>>> Tatsuo Ishii
>>>> SRA OSS, Inc. Japan
>>>> English: http://www.sraoss.co.jp/index_en.php
>>>> Japanese:http://www.sraoss.co.jp
>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>> is there any news on this topic? Any information that I can provide to help?
>>>>> Thanks again for your help.
>>>>>
>>>>>
>>>>> Kind regards
>>>>> Benjamin Firl
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 03/02/2018 09:33 AM, Benjamin Firl wrote:
>>>>>> We have seen problems with online recovery in pgpool-II version 3.6.7
>>>>>> (subaruboshi). But we updated to 3.7.2 before we ware able to reproduce
>>>>>> and analyze it. So I suspect the same thing happened in 3.6.7, but I am
>>>>>> not 100% sure.
>>>>>> Adding a new lowest node id is a relatively rare case in our setup.
>>>>>>
>>>>>>
>>>>>> Kind regards
>>>>>> Benjamin Firl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 03/02/2018 12:16 AM, Tatsuo Ishii wrote:
>>>>>>> I suspect it's a bug with 3.7.2. Have you seen similar issue with
>>>>>>> other Pgpool-II versions?
>>>>>>>
>>>>>>> Best regards,
>>>>>>> --
>>>>>>> Tatsuo Ishii
>>>>>>> SRA OSS, Inc. Japan
>>>>>>> English: http://www.sraoss.co.jp/index_en.php
>>>>>>> Japanese:http://www.sraoss.co.jp
>>>>>>>
>>>>>>>> our script for online recovery stage1 seems to get wrong arguments
>>>>>>>> sometimes and thus is failing.
>>>>>>>>
>>>>>>>> We log the arguments, so on a successful run it prints something like
>>>>>>>> that on the master node:
>>>>>>>>
>>>>>>>> recovery_1st_stage.sh
>>>>>>>> all args: /var/lib/postgresql/9.5/main 10.0.1.25
>>>>>>>> /var/lib/postgresql/9.5/main 5432
>>>>>>>> data: /var/lib/postgresql/9.5/main
>>>>>>>> recovery target: 10.0.1.25
>>>>>>>> recovery data: /var/lib/postgresql/9.5/main
>>>>>>>>
>>>>>>>>
>>>>>>>> If recovery fails, we get this:
>>>>>>>>
>>>>>>>> recovery_1st_stage.sh
>>>>>>>> all args: /var/lib/postgresql/9.5/main localhost 5432
>>>>>>>> data: /var/lib/postgresql/9.5/main
>>>>>>>> recovery target: localhost
>>>>>>>> recovery data: 5432
>>>>>>>> unable to recover myself....exiting
>>>>>>>> 2018-03-01 14:31:59 CET [10517-1] postgres at template1 ERROR:
>>>>>>>> pgpool_recovery failed
>>>>>>>> 2018-03-01 14:31:59 CET [10517-2] postgres at template1 STATEMENT: SELECT
>>>>>>>> pgpool_recovery('recovery_1st_stage.sh', 'localhost', '', '5432')
>>>>>>>>
>>>>>>>>
>>>>>>>> This seems to happen, if the new node has a lower id than the master
>>>>>>>> node. But normally adding a new node with lower id should not be a
>>>>>>>> problem, am I correct?
>>>>>>>> We are using PostgreSQL 9.5.10 and pgpool-II version 3.7.2 (amefuriboshi).
>>>>>>>> Thanks for your help.
>>>>>>>>
>>>>>>>>
>>>>>>>> Kind regards
>>>>>>>> Benjamin Firl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>
>>>>>>>> +++ Jetzt neu Wissensmanagement für Netzwerke +++
>>>>>>>>
>>>>>>>> +++ www.knodge.de +++
>>>>>>>>
>>>>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> www.wisit.com
>>>>>>>>
>>>>>>>> www.knodge.de
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> wisit media GmbH
>>>>>>>>
>>>>>>>> Ehrenbergstr. 11
>>>>>>>>
>>>>>>>> D-98693 Ilmenau
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> wisit media GmbH, Ehrenbergstr. 11, D-98693 Ilmenau
>>>>>>>>
>>>>>>>> Registergericht Jena HRB 512472
>>>>>>>>
>>>>>>>> Geschaeftsfuehrung: Dipl. Ing. Markus Duelli
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich
>>>>>>>>
>>>>>>>> erhalten haben, informieren Sie bitte sofort den Absender und vernichten
>>>>>>>>
>>>>>>>> Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
>>>>>>>>
>>>>>>>> dieser E-Mail ist nicht gestattet.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> pgpool-general mailing list
>>>>>>>> pgpool-general at pgpool.net
>>>>>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>>>> --
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>
>>>>> +++ Jetzt neu Wissensmanagement für Netzwerke +++
>>>>>
>>>>> +++ www.knodge.de +++
>>>>>
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> www.wisit.com
>>>>>
>>>>> www.knodge.de
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> wisit media GmbH
>>>>>
>>>>> Ehrenbergstr. 11
>>>>>
>>>>> D-98693 Ilmenau
>>>>>
>>>>> ---------------------------------------------------------------------------
>>>>>
>>>>> -
>>>>>
>>>>> wisit media GmbH, Ehrenbergstr. 11, D-98693 Ilmenau
>>>>>
>>>>> Registergericht Jena HRB 512472
>>>>>
>>>>> Geschaeftsfuehrung: Dipl. Ing. Markus Duelli
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich
>>>>>
>>>>> erhalten haben, informieren Sie bitte sofort den Absender und vernichten
>>>>>
>>>>> Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
>>>>>
>>>>> dieser E-Mail ist nicht gestattet.
>>>>>
>>>>> _______________________________________________
>>>>> pgpool-general mailing list
>>>>> pgpool-general at pgpool.net
>>>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>>> --
>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>> +++ Jetzt neu Wissensmanagement für Netzwerke +++
>>>
>>> +++ www.knodge.de +++
>>>
>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>
>>>
>>> --
>>>
>>> www.wisit.com
>>>
>>> www.knodge.de
>>>
>>>
>>>
>>>
>>>
>>> wisit media GmbH
>>>
>>> Ehrenbergstr. 11
>>>
>>> D-98693 Ilmenau
>>>
>>> ---------------------------------------------------------------------------
>>>
>>> -
>>>
>>> wisit media GmbH, Ehrenbergstr. 11, D-98693 Ilmenau
>>>
>>> Registergericht Jena HRB 512472
>>>
>>> Geschaeftsfuehrung: Dipl. Ing. Markus Duelli
>>>
>>>
>>>
>>>
>>>
>>> Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich
>>>
>>> erhalten haben, informieren Sie bitte sofort den Absender und vernichten
>>>
>>> Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
>>>
>>> dieser E-Mail ist nicht gestattet.
>>>
>>> _______________________________________________
>>> pgpool-general mailing list
>>> pgpool-general at pgpool.net
>>> http://www.pgpool.net/mailman/listinfo/pgpool-general
>
> --
> +++++++++++++++++++++++++++++++++++++++++++++++++
>
> +++ Jetzt neu Wissensmanagement für Netzwerke +++
>
> +++ www.knodge.de +++
>
> +++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
> --
>
> www.wisit.com
>
> www.knodge.de
>
>
>
>
>
> wisit media GmbH
>
> Ehrenbergstr. 11
>
> D-98693 Ilmenau
>
> ---------------------------------------------------------------------------
>
> -
>
> wisit media GmbH, Ehrenbergstr. 11, D-98693 Ilmenau
>
> Registergericht Jena HRB 512472
>
> Geschaeftsfuehrung: Dipl. Ing. Markus Duelli
>
>
>
>
>
> Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich
>
> erhalten haben, informieren Sie bitte sofort den Absender und vernichten
>
> Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
>
> dieser E-Mail ist nicht gestattet.
>
>
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general
--
+++++++++++++++++++++++++++++++++++++++++++++++++
+++ Jetzt neu Wissensmanagement für Netzwerke +++
+++ www.knodge.de +++
+++++++++++++++++++++++++++++++++++++++++++++++++
--
www.wisit.com
www.knodge.de
wisit media GmbH
Ehrenbergstr. 11
D-98693 Ilmenau
---------------------------------------------------------------------------
-
wisit media GmbH, Ehrenbergstr. 11, D-98693 Ilmenau
Registergericht Jena HRB 512472
Geschaeftsfuehrung: Dipl. Ing. Markus Duelli
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich
erhalten haben, informieren Sie bitte sofort den Absender und vernichten
Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
dieser E-Mail ist nicht gestattet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20180416/8eccedf9/attachment.html>
More information about the pgpool-general
mailing list