View Issue Details

IDProjectCategoryView StatusLast Update
0000314Pgpool-IIBugpublic2017-06-07 14:22
ReporternagataAssigned Tot-ishii 
PrioritynormalSeveritymajorReproducibilityhave not tried
Status resolvedResolutionopen 
Product Version3.5.8 
Target Version3.5.9Fixed in Version 
Summary0000314: kind mismatch error with extended protocol when close for portal is issued
DescriptionWith streaming-replication mode and extended protocol,
kind mismatch error occurs if Close (portal) messages
is sent before Bind.

 FATAL: failed to read kind from backend
 DETAIL: kind mismatch among backends.
  Possible last query was: "SELECT * FROM test_tbl"
  kind details are: 0[E: prepared statement "S_1" does not exist] 1[2]]

For details, see Steps To Reproduce.

Such Close messages before query execution can be issued by JDBC.
For details, see comments about processDeadParsedQueries and
processDeadPortals.

https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/core/v3/QueryExecutorImpl.java


[Analysis]

When Close is issued from frontend, Pgpool-II send this message to both the primary
and the standby by rewriting query_context->where_to_send[]. Due to this, Bind
message following is also sent to both backends, but Parse was sent only one backend,
so the error occurs.


 LOG: DB node id: 1 backend pid: 32586 statement: Parse: SELECT * FROM test_tbl
 LOG: DB node id: 1 backend pid: 32586 statement: B message
 LOG: DB node id: 1 backend pid: 32586 statement: Execute: SELECT * FROM test_tbl
 LOG: DB node id: 1 backend pid: 32586 statement: C message
 LOG: DB node id: 0 backend pid: 32585 statement: C message
 LOG: DB node id: 1 backend pid: 32586 statement: B message
 LOG: DB node id: 0 backend pid: 32585 statement: B message
 LOG: DB node id: 1 backend pid: 32586 statement: Execute: SELECT * FROM test_tbl
 LOG: DB node id: 0 backend pid: 32585 statement: Execute: SELECT * FROM test_tbl
 WARNING: packet kind of backend 0 ['E'] does not match with master/majority nodes packet kind ['2']
 FATAL: failed to read kind from backend
 DETAIL: kind mismatch among backends. Possible last query was: "SELECT * FROM test_tbl" kind details are : 0[E: prepared statement "S_1" does not exist] 1[2]
 HINT: check data consistency among db nodes


[Patch]

The attached is a simple patch to fix it. In this fix,
Close message is sent to only the load blance node when this is Close for portals.
Instead when this is Close for statements, this is sent to both nodes as same as
before.
Steps To ReproduceUse pgproto
--------------------------------------------

'Q' "DROP TABLE IF EXISTS test_tbl"
'Q' "CREATE TABLE test_tbl(i INT)"
'Q' "INSERT INTO test_tbl VALUES(1)"
'Y'
'Y'
'Y'

'P' "S_1" "SELECT * FROM test_tbl" 0
'B' "C_1" "S_1" 0 0 0
'E' "C_1" 0
'S'
'Y'

'C' 'P' "C_1"

'B' "C_2" "S_1" 0 0 0
'E' "C_2" 0
'S'
'Y'

'X'
--------------------------------------------
TagsNo tags attached.

Activities

nagata

2017-06-06 20:29

developer  

fix_kind_mismatch.patch (621 bytes)
diff --git a/src/protocol/pool_proto_modules.c b/src/protocol/pool_proto_modules.c
index 93f309b..d61c76b 100644
--- a/src/protocol/pool_proto_modules.c
+++ b/src/protocol/pool_proto_modules.c
@@ -1553,7 +1553,7 @@ POOL_STATUS Close(POOL_CONNECTION *frontend, POOL_CONNECTION_POOL *backend,
 	{
 		POOL_PENDING_MESSAGE *pmsg;
 
-		if (session_context->load_balance_node_id != PRIMARY_NODE_ID)
+		if (session_context->load_balance_node_id != PRIMARY_NODE_ID && *contents == 'S')
 		{
 			query_context->where_to_send[PRIMARY_NODE_ID] = true;
 			query_context->where_to_send[session_context->load_balance_node_id] = true;

t-ishii

2017-06-07 11:46

developer   ~0001533

Yes, the bind message complains that the prepared statement S_1 is missing on node 0.
However I think the proposed fix is not quite correct. Suppose parse_before_bind() was executed before and client wants to explicitly close the portal to reuse the same named portal, your patch leaves the portal on the primary node and will cause the error "the portal is already used".

Proper fix would be: if Pgpool-II rewrite the where_to_send map data, it should be restored later on. I will work on in the direction.

BTW,

'Q' "DROP TABLE IF EXISTS test_tbl"
'Q' "CREATE TABLE test_tbl(i INT)"
'Q' "INSERT INTO test_tbl VALUES(1)"
'Y'
'Y'
'Y'

are not quite correct from the frontend/backend protocol's view. Client should send next query upon receiving read query for message. The should be:

'Q' "DROP TABLE IF EXISTS test_tbl"
'Y'
'Q' "CREATE TABLE test_tbl(i INT)"
'Y'
'Q' "INSERT INTO test_tbl VALUES(1)"
'Y'

t-ishii

2017-06-07 14:21

developer   ~0001534

Fix committed.

Issue History

Date Modified Username Field Change
2017-06-06 20:29 nagata New Issue
2017-06-06 20:29 nagata File Added: fix_kind_mismatch.patch
2017-06-06 21:12 nagata Steps to Reproduce Updated View Revisions
2017-06-06 21:17 nagata Description Updated View Revisions
2017-06-07 11:30 t-ishii Assigned To => t-ishii
2017-06-07 11:30 t-ishii Status new => assigned
2017-06-07 11:46 t-ishii Note Added: 0001533
2017-06-07 11:54 t-ishii Target Version => 3.5.9
2017-06-07 14:21 t-ishii Note Added: 0001534
2017-06-07 14:22 t-ishii Status assigned => resolved