View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000209 | Pgpool-II | Bug | public | 2016-06-27 23:08 | 2016-12-20 14:05 |
| Reporter | ochedru | Assigned To | t-ishii | ||
| Priority | normal | Severity | major | Reproducibility | always |
| Status | closed | Resolution | open | ||
| Summary | 0000209: Query cache invalidation performed too late | ||||
| Description | Query cache invalidation (done by pool_handle_query_cache), is performed from ReadyForQuery() AFTER the frontend has been notified (see send_ready flag). As a consequence, the frontend can continue querying on another connection and get inconsistent results. The frontend must be notified only after the cache is in a consistent state. | ||||
| Steps To Reproduce | See the attached testcase using Java 8/JDBC. The expected output is like: java.lang.RuntimeException: Got 133 but expected 134 at Main.lambda$main$2(Main.java:104) at Main$Test.run(Main.java:38) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) | ||||
| Tags | No tags attached. | ||||
|
|
|
|
|
Uploaded new test-case with call to "executor.shutdown()": if the test passes we want the program to exit successfully. |
|
|
|
|
|
I could make the test pass by moving the block below at the end of function ReadyForQuery. Not sure if there are possible side effects, but my frontend application is working fine now. if (send_ready) { pool_write(frontend, "Z", 1); if (MAJOR(backend) == PROTO_MAJOR_V3) { len = htonl(len); pool_write(frontend, &len, sizeof(len)); pool_write(frontend, &state, 1); } pool_flush(frontend); } |
|
|
FYI, actually the application is still not working fine; I have problems with cache auto invalidation after a transaction commit: query_context->temp_cache is null sometimes, thus preventing pool_handle_query_cache() to invalidate the modified table. I'm trying to setup a testcase for this one. |
|
|
I think your proposal does not solve the problem entirely. Even if Pgpool-II invalidates query cache right after "Command Complete" message (which is sent from PostgreSQL *before* "Ready for query"), it's always possible that other client can see the old cache *after* Command Complete message and *before* the cache invalidation. We cannot eliminate the window unless DML execution and cache invalidation is an atomic operation, which is not possible in our architecture. To implement that, the query cache needs to be built-in in PostgreSQL. |
|
|
I agree the whole operation cannot be atomic. However, the client application usually has some sort of synchronization built-in: for example, some transactions are done sequentially (c.f. test case). IMHO, ending the transaction with the cache invalidated is better because the inconsistency window is smaller. However the fix does not work because sometimes invalidation is not performed: query_context->temp_cache is null and I do not know why. |
|
|
I have decided we are not going change the current behavior. Sorry. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2016-06-27 23:08 | ochedru | New Issue | |
| 2016-06-27 23:08 | ochedru | File Added: test_cache_invalidation.zip | |
| 2016-06-27 23:47 | ochedru | Note Added: 0000869 | |
| 2016-06-27 23:47 | ochedru | File Added: test_cache_invalidation_2.zip | |
| 2016-06-27 23:51 | ochedru | Note Added: 0000870 | |
| 2016-07-01 15:27 | t-ishii | Assigned To | => t-ishii |
| 2016-07-01 15:27 | t-ishii | Status | new => assigned |
| 2016-07-01 17:19 | ochedru | Note Added: 0000876 | |
| 2016-08-09 10:43 | t-ishii | Note Added: 0000971 | |
| 2016-08-09 10:43 | t-ishii | Status | assigned => feedback |
| 2016-08-17 17:50 | ochedru | Note Added: 0001007 | |
| 2016-08-17 17:50 | ochedru | Status | feedback => assigned |
| 2016-12-20 14:05 | t-ishii | Note Added: 0001241 | |
| 2016-12-20 14:05 | t-ishii | Status | assigned => closed |
