View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000467 | Pgpool-II | Bug | public | 2019-02-20 14:06 | 2019-08-14 16:12 |
| Reporter | max3903 | Assigned To | pengbo | ||
| Priority | urgent | Severity | major | Reproducibility | always |
| Status | assigned | Resolution | open | ||
| Platform | Linux | OS | Oracle Linux | OS Version | 7 |
| Product Version | 4.0.2 | ||||
| Target Version | 4.0.6 | ||||
| Summary | 0000467: Encoding issue with bytea field and PostgreSQL 11 | ||||
| Description | I am using 3 servers: Application Server: * Oracle Linux 7 * Odoo 11 * Python 3.6.6 * Psycopg2 2.7.5 PGPool Server: * Oracle Linux 7 * PGPool 4.0.2 from the yum repository DB Server: * Oracle Linux 7 * PostgreSQL 11 | ||||
| Steps To Reproduce | Odoo is using binary field to upload files: https://github.com/odoo/odoo/blob/12.0/addons/base_import/models/base_import.py#L116 and psycopg2 store it in a bytea attribute: https://github.com/odoo/odoo/blob/12.0/odoo/fields.py#L1768 Any uploaded file is not stored correctly in the database when using PGPool. If I connect Odoo directly to PostgreSQL, all my scenario work. For example, when importing data from a csv file, the file is encoded and stored in the database. Then the first lines get displayed for preview and column-to-field matching. See screenshot attached below. Another example I have is that the value of a bytea attribute is decoded and I get a "Invalid padding" error: File "/usr/lib64/python3.6/base64.py", line 87, in b64decode return binascii.a2b_base64(s) binascii.Error: Incorrect padding | ||||
| Tags | pgpool-II | ||||
|
|
|
|
|
Attached is my pgpool.conf |
|
|
Hello, What can we do to get someone's attention to his issue and work on it? Thanks. |
|
|
@pengbo If I can help you in any way, please let me know. Thanks! Have a great day. Max |
|
|
Could you show me pgpool.log? |
|
|
Sure. See attached with log levels at debug1. Let me know if you need more debug info. |
|
|
Thank you. I could not see any errors in pgpool.log. Could you set "log_client_messages = on" to log client messages from application? |
|
|
Please find attached pgpool.log with log_client_messages = on |
|
|
PGPool receives: 2019-03-13T09:52:46-05:00 bap142aq pgpool[12244]: [1386-2] 2019-03-13 09:52:46: pid 12244: DETAIL: query: "UPDATE "base_import_import" SET 2019-03-13T09:52:46-05:00 bap142aq pgpool[12244]: [1386-3] "file"='\x69642c6e616d652c5638206c6f67696e2c6c6f67696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32392c414945534849412057414c4b45522c4157414c4b45522c6177616c6b65720d0a5f5f6578706f72745f5f2e7265735f75736572735f32372c414c4d412053494c56412c4153494c56412c6173696c76610d0a5f5f6578706f72745f5f2e7265735f75736572735f32352c414d42414c205155494e4e2c415155494e4e2c617175696e6e0d0a5f5f6578706f72745f5f2e7265735f75736572735f34365f35393364383738362c414e41204c414755414e2c414c414755414e2c616c616775616e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32312c4341524c4f532049524941532c4349524941532c6369726961730d0a5f5f6578706f72745f5f2e7265735f75736572735f33335f32363534333336362c434841442050414c4d51554953542c4350414c4d51554953542c6370616c6d71756973740d0a5f5f6578706f72745f5f2e7265735f75736572735f3133385f32353365373764382c4368726973204572696b73656e2c636572696b73656e40657463632e636f6d2c6572696b73656e6335383738380 2019-03-13T09:52:46-05:00 bap142aq pgpool[12244]: [1386-4] d0a5f5f6578706f72745f5f2e7265735f75736572735f3136315f37323566383065662c436f6c6279204b656174682c636b6561746840657463632e636f6d2c6b656174686330313031300d0a5f5f6578706f72745f5f2e7265735f75736572735f38392c444149535920415252454755494e2c44415252454755494e2c64617272656775696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31302c44454e495452412042524f574e2c4442524f574e2c6462726f776e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31362c444556494e20454c4c49532c44454c4c49532c64656c6c69730d0a5f5f6578706f72745f5f2e7265735f75736572735f32362c454c495a4142455448205348454548414e2c455348454548414e2c657368656568616e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32302c46454c455349412048454e4e4553592c4648454e4e4553592c6668656e6e6573790d0a5f5f6578706f72745f5f2e7265735f75736572735f362c48435452412041646d696e2c68637472615f61646d696e2c68637472615f61646d696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f355f3465363139366 2019-03-13T09:52:46-05:00 bap142aq pgpool[12244]: [1386-5] 2382c48435452412053757065727669736f722c68637472615f73757065727669736f722c68637472615f73757065727669736f720d0a5f5f6578706f72745f5f2e7265735f75736572735f32322c4a41435155454c494e45204d4f52454e4f2c4a4d4f52454e4f2c6a6d6f72656e6f0d0a5f5f6578706f72745f5f2e7265735f75736572735f33312c4a454e4e49464552205752494748542c4a5752494748542c6a7772696768740d0a5f5f6578706f72745f5f2e7265735f75736572735f32332c4a455353494341204f5245534b4f564943482c4a4f5245534b4f564943482c6a6f7265736b6f766963680d0a5f5f6578706f72745f5f2e7265735f75736572735f31332c4a4f4e415448414e204441484c2c4a4441484c2c6a6461686c0d0a5f5f6578706f72745f5f2e7265735f75736572735f37355f32363830653661302c4b424c4f4f44574f5254482c4b424c4f4f44574f5254482c6b626c6f6f64776f7274680d0a5f5f6578706f72745f5f2e7265735f75736572735f392c4b454e4e45544820424f57454e532c4b424f57454e532c6b626f77656e730d0a5f5f6578706f72745f5f2e7265735f75736572735f31322c4c455345414e204355544e4 2019-03-13T09:52:46-05:00 bap142aq pgpool[12244]: [1386-6] f2c4c4355544e4f2c6c6375746e6f0d0a5f5f6578706f72745f5f2e7265735f75736572735f31382c4d494348454c4c4520474f4e5a414c45532c4d474f4e5a414c45532c6d676f6e7a616c65730d0a5f5f6578706f72745f5f2e7265735f75736572735f31342c50415452494349412044454c4741444f2c4d44454c4741444f2c6d64656c6761646f0d0a5f5f6578706f72745f5f2e7265735f75736572735f31392c5048594c4c4953204752414e542c504752414e542c706772616e740d0a5f5f6578706f72745f5f2e7265735f75736572735f31312c52454e454520434f4f4b2c52434f4f4b2c72636f6f6b0d0a5f5f6578706f72745f5f2e7265735f75736572735f33355f39333235653931622c5253494d532c5253494d532c7273696d730d0a5f5f6578706f72745f5f2e7265735f75736572735f32382c534849524c4559205350414e4e2c535350414e4e2c737370616e6e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31352c54414b495348412044554741532c5444554741532c7464756761730d0a5f5f6578706f72745f5f2e7265735f75736572735f33302c54414d454b4120574849544d4952452c574849544d4952455431383 2019-03-13T09:52:46-05:00 bap142aq pgpool[12244]: [1386-7] 934322c776869746d6972657431383934320d0a5f5f6578706f72745f5f2e7265735f75736572735f382c5449412042494c4c495249532c5443415252454c4c2c7463617272656c6c0d0a5f5f6578706f72745f5f2e7265735f75736572735f3136385f38366665643266382c56696e6f642042616a6a7572692c5642414a4a5552492c62616a6a7572697630313031300d0a5f5f6578706f72745f5f2e7265735f75736572735f32342c594144484952412050415252412c5950454345524f2c7970656365726f0d0a5f5f6578706f72745f5f2e7265735f75736572735f3133392c59656461204461766964736f6e2c796461766964736f6e40657463632e636f6d2c6461766964736f6e7930313031300d0a'::bytea,"file_name"='res users - res users.utf-8.csv',"file_type"='text/csv',"write_uid"=1,"write_date"=(now() at time zone 'UTC') WHERE id IN (26)" but PostgreSQL receives: Mar 13 09:51:48 bap144aq postgres[8600]: [55-1] 2019-03-13 09:51:48.748 CDT [8600] LOG: duration: 2.084 ms statement: UPDATE "base_import_import" SET "file" = Mar 13 09:51:48 bap144aq postgres[8600]: [55-2] '\\x69642c6e616d652c5638206c6f67696e2c6c6f67696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32392c414945534849412057414c4b45522c4157414c4b45522c6177616c6b65720d0a5f5f6578706f72745f5f2e7265735f75736572735f32372c414c4d412053494c56412c4153494c56412c6173696c76610d0a5f5f6578706f72745f5f2e7265735f75736572735f32352c414d42414c205155494e4e2c415155494e4e2c617175696e6e0d0a5f5f6578706f72745f5f2e7265735f75736572735f34365f35393364383738362c414e41204c414755414e2c414c414755414e2c616c616775616e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32312c4341524c4f532049524941532c4349524941532c6369726961730d0a5f5f6578706f72745f5f2e7265735f75736572735f33335f32363534333336362c434841442050414c4d51554953542c4350414c4d51554953542c6370616c6d71756973740d0a5f5f6578706f72745f5f2e7265735f75736572735f3133385f32353365373764382c4368726973204572696b73656e2c636572696b73656e40657463632e636f6d2c6572696b73656e6335383738380d0a5f5 Mar 13 09:51:48 bap144aq postgres[8600]: [55-3] f6578706f72745f5f2e7265735f75736572735f3136315f37323566383065662c436f6c6279204b656174682c636b6561746840657463632e636f6d2c6b656174686330313031300d0a5f5f6578706f72745f5f2e7265735f75736572735f38392c444149535920415252454755494e2c44415252454755494e2c64617272656775696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31302c44454e495452412042524f574e2c4442524f574e2c6462726f776e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31362c444556494e20454c4c49532c44454c4c49532c64656c6c69730d0a5f5f6578706f72745f5f2e7265735f75736572735f32362c454c495a4142455448205348454548414e2c455348454548414e2c657368656568616e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32302c46454c455349412048454e4e4553592c4648454e4e4553592c6668656e6e6573790d0a5f5f6578706f72745f5f2e7265735f75736572735f362c48435452412041646d696e2c68637472615f61646d696e2c68637472615f61646d696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f355f34653631393662382c4 Mar 13 09:51:48 bap144aq postgres[8600]: [55-4] 8435452412053757065727669736f722c68637472615f73757065727669736f722c68637472615f73757065727669736f720d0a5f5f6578706f72745f5f2e7265735f75736572735f32322c4a41435155454c494e45204d4f52454e4f2c4a4d4f52454e4f2c6a6d6f72656e6f0d0a5f5f6578706f72745f5f2e7265735f75736572735f33312c4a454e4e49464552205752494748542c4a5752494748542c6a7772696768740d0a5f5f6578706f72745f5f2e7265735f75736572735f32332c4a455353494341204f5245534b4f564943482c4a4f5245534b4f564943482c6a6f7265736b6f766963680d0a5f5f6578706f72745f5f2e7265735f75736572735f31332c4a4f4e415448414e204441484c2c4a4441484c2c6a6461686c0d0a5f5f6578706f72745f5f2e7265735f75736572735f37355f32363830653661302c4b424c4f4f44574f5254482c4b424c4f4f44574f5254482c6b626c6f6f64776f7274680d0a5f5f6578706f72745f5f2e7265735f75736572735f392c4b454e4e45544820424f57454e532c4b424f57454e532c6b626f77656e730d0a5f5f6578706f72745f5f2e7265735f75736572735f31322c4c455345414e204355544e4f2c4c4 Mar 13 09:51:48 bap144aq postgres[8600]: [55-5] 355544e4f2c6c6375746e6f0d0a5f5f6578706f72745f5f2e7265735f75736572735f31382c4d494348454c4c4520474f4e5a414c45532c4d474f4e5a414c45532c6d676f6e7a616c65730d0a5f5f6578706f72745f5f2e7265735f75736572735f31342c50415452494349412044454c4741444f2c4d44454c4741444f2c6d64656c6761646f0d0a5f5f6578706f72745f5f2e7265735f75736572735f31392c5048594c4c4953204752414e542c504752414e542c706772616e740d0a5f5f6578706f72745f5f2e7265735f75736572735f31312c52454e454520434f4f4b2c52434f4f4b2c72636f6f6b0d0a5f5f6578706f72745f5f2e7265735f75736572735f33355f39333235653931622c5253494d532c5253494d532c7273696d730d0a5f5f6578706f72745f5f2e7265735f75736572735f32382c534849524c4559205350414e4e2c535350414e4e2c737370616e6e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31352c54414b495348412044554741532c5444554741532c7464756761730d0a5f5f6578706f72745f5f2e7265735f75736572735f33302c54414d454b4120574849544d4952452c574849544d4952455431383934322 Mar 13 09:51:48 bap144aq postgres[8600]: [55-6] c776869746d6972657431383934320d0a5f5f6578706f72745f5f2e7265735f75736572735f382c5449412042494c4c495249532c5443415252454c4c2c7463617272656c6c0d0a5f5f6578706f72745f5f2e7265735f75736572735f3136385f38366665643266382c56696e6f642042616a6a7572692c5642414a4a5552492c62616a6a7572697630313031300d0a5f5f6578706f72745f5f2e7265735f75736572735f32342c594144484952412050415252412c5950454345524f2c7970656365726f0d0a5f5f6578706f72745f5f2e7265735f75736572735f3133392c59656461204461766964736f6e2c796461766964736f6e40657463632e636f6d2c6461766964736f6e7930313031300d0a'::"bytea", "file_name" = 'res users - res users.utf-8.csv', "file_type" = 'text/csv', "write_uid" = 1, "write_date" = "pg_catalog"."timezone"('UTC',"pg_catalog"."timestamptz"('2019-03-13 09:51:48.737273-05'::text)) WHERE "id" IN (26) Without PGPool, PostgreSQL receives: Mar 13 09:56:14 bap144aq postgres[9343]: [45-1] 2019-03-13 09:56:14.306 CDT [9343] LOG: duration: 0.825 ms statement: UPDATE "base_import_import" SET Mar 13 09:56:14 bap144aq postgres[9343]: [45-2] "file"='\x69642c6e616d652c5638206c6f67696e2c6c6f67696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32392c414945534849412057414c4b45522c4157414c4b45522c6177616c6b65720d0a5f5f6578706f72745f5f2e7265735f75736572735f32372c414c4d412053494c56412c4153494c56412c6173696c76610d0a5f5f6578706f72745f5f2e7265735f75736572735f32352c414d42414c205155494e4e2c415155494e4e2c617175696e6e0d0a5f5f6578706f72745f5f2e7265735f75736572735f34365f35393364383738362c414e41204c414755414e2c414c414755414e2c616c616775616e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32312c4341524c4f532049524941532c4349524941532c6369726961730d0a5f5f6578706f72745f5f2e7265735f75736572735f33335f32363534333336362c434841442050414c4d51554953542c4350414c4d51554953542c6370616c6d71756973740d0a5f5f6578706f72745f5f2e7265735f75736572735f3133385f32353365373764382c4368726973204572696b73656e2c636572696b73656e40657463632e636f6d2c6572696b73656e6335383738380 Mar 13 09:56:14 bap144aq postgres[9343]: [45-3] d0a5f5f6578706f72745f5f2e7265735f75736572735f3136315f37323566383065662c436f6c6279204b656174682c636b6561746840657463632e636f6d2c6b656174686330313031300d0a5f5f6578706f72745f5f2e7265735f75736572735f38392c444149535920415252454755494e2c44415252454755494e2c64617272656775696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31302c44454e495452412042524f574e2c4442524f574e2c6462726f776e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31362c444556494e20454c4c49532c44454c4c49532c64656c6c69730d0a5f5f6578706f72745f5f2e7265735f75736572735f32362c454c495a4142455448205348454548414e2c455348454548414e2c657368656568616e0d0a5f5f6578706f72745f5f2e7265735f75736572735f32302c46454c455349412048454e4e4553592c4648454e4e4553592c6668656e6e6573790d0a5f5f6578706f72745f5f2e7265735f75736572735f362c48435452412041646d696e2c68637472615f61646d696e2c68637472615f61646d696e0d0a5f5f6578706f72745f5f2e7265735f75736572735f355f3465363139366 Mar 13 09:56:14 bap144aq postgres[9343]: [45-4] 2382c48435452412053757065727669736f722c68637472615f73757065727669736f722c68637472615f73757065727669736f720d0a5f5f6578706f72745f5f2e7265735f75736572735f32322c4a41435155454c494e45204d4f52454e4f2c4a4d4f52454e4f2c6a6d6f72656e6f0d0a5f5f6578706f72745f5f2e7265735f75736572735f33312c4a454e4e49464552205752494748542c4a5752494748542c6a7772696768740d0a5f5f6578706f72745f5f2e7265735f75736572735f32332c4a455353494341204f5245534b4f564943482c4a4f5245534b4f564943482c6a6f7265736b6f766963680d0a5f5f6578706f72745f5f2e7265735f75736572735f31332c4a4f4e415448414e204441484c2c4a4441484c2c6a6461686c0d0a5f5f6578706f72745f5f2e7265735f75736572735f37355f32363830653661302c4b424c4f4f44574f5254482c4b424c4f4f44574f5254482c6b626c6f6f64776f7274680d0a5f5f6578706f72745f5f2e7265735f75736572735f392c4b454e4e45544820424f57454e532c4b424f57454e532c6b626f77656e730d0a5f5f6578706f72745f5f2e7265735f75736572735f31322c4c455345414e204355544e4 Mar 13 09:56:14 bap144aq postgres[9343]: [45-5] f2c4c4355544e4f2c6c6375746e6f0d0a5f5f6578706f72745f5f2e7265735f75736572735f31382c4d494348454c4c4520474f4e5a414c45532c4d474f4e5a414c45532c6d676f6e7a616c65730d0a5f5f6578706f72745f5f2e7265735f75736572735f31342c50415452494349412044454c4741444f2c4d44454c4741444f2c6d64656c6761646f0d0a5f5f6578706f72745f5f2e7265735f75736572735f31392c5048594c4c4953204752414e542c504752414e542c706772616e740d0a5f5f6578706f72745f5f2e7265735f75736572735f31312c52454e454520434f4f4b2c52434f4f4b2c72636f6f6b0d0a5f5f6578706f72745f5f2e7265735f75736572735f33355f39333235653931622c5253494d532c5253494d532c7273696d730d0a5f5f6578706f72745f5f2e7265735f75736572735f32382c534849524c4559205350414e4e2c535350414e4e2c737370616e6e0d0a5f5f6578706f72745f5f2e7265735f75736572735f31352c54414b495348412044554741532c5444554741532c7464756761730d0a5f5f6578706f72745f5f2e7265735f75736572735f33302c54414d454b4120574849544d4952452c574849544d4952455431383 Mar 13 09:56:14 bap144aq postgres[9343]: [45-6] 934322c776869746d6972657431383934320d0a5f5f6578706f72745f5f2e7265735f75736572735f382c5449412042494c4c495249532c5443415252454c4c2c7463617272656c6c0d0a5f5f6578706f72745f5f2e7265735f75736572735f3136385f38366665643266382c56696e6f642042616a6a7572692c5642414a4a5552492c62616a6a7572697630313031300d0a5f5f6578706f72745f5f2e7265735f75736572735f32342c594144484952412050415252412c5950454345524f2c7970656365726f0d0a5f5f6578706f72745f5f2e7265735f75736572735f3133392c59656461204461766964736f6e2c796461766964736f6e40657463632e636f6d2c6461766964736f6e7930313031300d0a'::bytea,"file_name"='res users - res users.utf-8.csv',"file_type"='text/csv',"write_uid"=1,"write_date"=(now() at time zone 'UTC') WHERE id IN (26) |
|
|
Seems like pgpool is adding a \ |
|
|
It seems it only happens in replication mode, when a query with a timestamp column. I maybe a bug of pgpool's query rewriting. I will check it. |
|
|
Thank you! |
|
|
@pengbo Is there anything I can do to help and speed up the release of a fix? Thank you! |
|
|
I am working on this one. I will share you the patch when I completed. |
|
|
Hi @pengbo, Any update? |
|
|
Hi @pengbo, How much would you charge me to fix this bug by June 15th, 2019? We need to get this fixed here. Thanks! |
|
|
Sorry for late response. I was busy last time. I will fix it by the end of next week. Now I am thinking that to get the config of "standard_conforming_strings" and put it to shared relation cache to deal with '\'. |
|
|
End of next week is fine for me. Thank you. Let me know when you want me to test. |
|
|
Sorry for late response. I have fixed this issue: https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commitdiff;h=48a45fc9d6e56eb3025cf24bec80bf9659796e3e |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2019-02-20 14:06 | max3903 | New Issue | |
| 2019-02-20 14:06 | max3903 | File Added: Screenshot from 2019-02-19 22-59-42.png | |
| 2019-02-20 14:31 | max3903 | File Added: pgpool.conf | |
| 2019-02-20 14:31 | max3903 | Note Added: 0002396 | |
| 2019-02-20 14:31 | max3903 | Tag Attached: pgpool-II | |
| 2019-02-21 03:57 | max3903 | Note Added: 0002397 | |
| 2019-02-25 09:32 | administrator | Assigned To | => pengbo |
| 2019-02-25 09:32 | administrator | Status | new => assigned |
| 2019-02-25 09:32 | administrator | Description Updated | |
| 2019-02-25 09:32 | administrator | Steps to Reproduce Updated | |
| 2019-02-28 02:36 | max3903 | Note Added: 0002409 | |
| 2019-03-01 13:51 | pengbo | Note Added: 0002415 | |
| 2019-03-02 01:02 | max3903 | File Added: pgpool.log | |
| 2019-03-02 01:02 | max3903 | Note Added: 0002416 | |
| 2019-03-03 18:00 | pengbo | Note Added: 0002417 | |
| 2019-03-12 01:14 | max3903 | File Added: pgpool-2.log | |
| 2019-03-12 01:14 | max3903 | Note Added: 0002429 | |
| 2019-03-13 23:58 | max3903 | Note Added: 0002430 | |
| 2019-03-13 23:59 | max3903 | Note Added: 0002431 | |
| 2019-03-25 18:10 | pengbo | Note Added: 0002456 | |
| 2019-03-30 03:56 | max3903 | Note Added: 0002485 | |
| 2019-04-10 07:07 | max3903 | Note Added: 0002540 | |
| 2019-04-10 09:18 | pengbo | Note Added: 0002541 | |
| 2019-05-11 03:10 | max3903 | Note Added: 0002596 | |
| 2019-05-27 23:19 | max3903 | Note Added: 0002638 | |
| 2019-05-30 01:42 | pengbo | Note Added: 0002640 | |
| 2019-05-30 02:00 | max3903 | Note Added: 0002641 | |
| 2019-06-07 17:34 | pengbo | Note Added: 0002644 | |
| 2019-08-14 16:12 | pengbo | Target Version | => 4.0.6 |