[pgpool-general: 1001] Re: using prepared statements when memory_cache_enabled=on

Lonni J Friedman netllama at gmail.com
Fri Sep 14 07:00:30 JST 2012


I don't think you can assert that pgpool is behaving completely sane,
when I have a failure that only reproduces when caching is enabled.  I
fully understand that developers need reproducible test cases.  I
never insisted that you fix this, with nothing useful to work with.
My expectation has been that we work together to at least troubleshoot
what is going on.

I'd love to give you a reproducible test case.  I spent close to 6
hours working on this in the past 24 hours trying to isolate it down
to something trivial, and I'm not getting anywhere.  I haven't found
any way to trigger the failure that doesn't require over a dozen
different tables, and a few dozen SQL queries.  If you really want, I
can give you a dump with all the data, plus the php required to
trigger it, but its not going to be pretty to setup.

Again the error that we're seeing is as follows:
server sent data ("D" message) without prior row description ("T"
message)Failure of query.


Does pgpool's caching treat "D" message data or "T" message
descriptions differently in a special way?

If I set log_statement=on and debug_level=1 and capture everything
that is logged when this fails, will that provide anything useful?  By
useful, I do *NOT* neccesarily mean something that would let you fix
whatever is wrong, but at least a clue for further investigation?

Is there any other pgpool.conf option that I can enable or change that
might be useful in determining what is going wrong?


On Wed, Sep 12, 2012 at 2:55 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
> I think no one can debug pgpool when it behaves completely sane.
> Again, developers need reproducible test case. What you can do is, try
> to find the case.
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
>
>> Yes, I understand that, but that's not a valid excuse for ignoring us.
>>  I asked how we can debug this, and received no reply.  It looks like
>> pgpool is doing something wrong.  How can I debug what its doing?
>>
>> On Wed, Sep 12, 2012 at 2:36 PM, Tatsuo Ishii <ishii at postgresql.org> wrote:
>>> I guess she stucks because she doesn't know how to reproduce the problem.
>>> --
>>> Tatsuo Ishii
>>> SRA OSS, Inc. Japan
>>> English: http://www.sraoss.co.jp/index_en.php
>>> Japanese: http://www.sraoss.co.jp
>>>
>>>> The lack of response isn't comforting.  There's clearly a bug
>>>> somewhere, and the only way it reproduces is when using pgpool with
>>>> memcache enabled.  Does no one on the pgpool team care about this
>>>> problem?
>>>>
>>>> On Mon, Sep 10, 2012 at 7:33 AM, Lonni J Friedman <netllama at gmail.com> wrote:
>>>>> Nozomi, we'd appreciate some input & assistance.  thanks!
>>>>>
>>>>> On Thu, Sep 6, 2012 at 10:12 AM, Greg Swallow <gswallow at exacttarget.com> wrote:
>>>>>> Yep.
>>>>>>
>>>>>> On Sep 6, 2012, at 1:10 PM, Lonni J Friedman wrote:
>>>>>>
>>>>>>> Its somewhat encouraging knowing its not just me.  Are you seeing the
>>>>>>> same error that I'm seeing?
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Sep 6, 2012 at 6:53 AM, Greg Swallow <gswallow at exacttarget.com> wrote:
>>>>>>>> I'm in the same boat, with a Rails app.  I can't reproduce it running the same query in psql, but once I turned the memory cache off, I stopped having this issue. I'm using membase and moxi, rather than memcached.  It's wire-format compatible from what I can tell.  PostgreSQL 9.1.4.  I can possibly help debug.
>>>>>>>>
>>>>>>>> On Sep 5, 2012, at 5:26 PM, Lonni J Friedman wrote:
>>>>>>>>
>>>>>>>>> On Tue, Sep 4, 2012 at 9:30 PM, Nozomi Anzai <anzai at sraoss.co.jp> wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>>> Greetings,
>>>>>>>>>>> I'm running 3.2.0 with memory_cache_enabled=on and memqcache_method =
>>>>>>>>>>> 'memcached', and I'm seeing really bizarre errors when using prepared
>>>>>>>>>>> statements.  Basically any query that should use the prepared
>>>>>>>>>>> statement is failing with:
>>>>>>>>>>> server sent data ("D" message) without prior row description ("T" message)
>>>>>>>>>>>
>>>>>>>>>>> This only happens once the data has been cached by pgpool (with the
>>>>>>>>>>> memcache server).  If I set memory_cache_enabled=off, then it works
>>>>>>>>>>> fine.  I'm guessing that maybe the problem is that the prepared
>>>>>>>>>>> statement isn't getting cached, so the queries which depend on the
>>>>>>>>>>> prepared statement fail?
>>>>>>>>>>>
>>>>>>>>>>> Is this expected behavior?  If so, is there a work around that doesn't
>>>>>>>>>>> require me to not cache the tables that the prepared statement
>>>>>>>>>>> reference?
>>>>>>>>>>
>>>>>>>>>> I couldn't reproduce it by the following ways:
>>>>>>>>>>
>>>>>>>>>> 1) the attached simple Java program: pgsql.java (only executing one SELECT)
>>>>>>>>>> 2) pgbench -M extended -f bench.sql
>>>>>>>>>>
>>>>>>>>>> Which version is your PostgreSQL? I tested with pgpool-II 3.2.0
>>>>>>>>>> (officially released), PostgreSQL 9.1.0, and memcached 1.4.7.
>>>>>>>>>
>>>>>>>>> postgresql-9.1.5
>>>>>>>>> memcached-1.4.10
>>>>>>>>>
>>>>>>>>> However, it looks like I was mistaken when I stated that this was
>>>>>>>>> triggered with prepared statements.  I tried to put together a trivial
>>>>>>>>> test case (similar to what you created), and now I can't replicate it.
>>>>>>>>> So whatever is going wrong only seems to happen inside of my huge,
>>>>>>>>> complex web app.    FWIW, I'm using PHP, not java.  All I know is that
>>>>>>>>> this reproduces 100% of the time whenever memory_cache_enabled=on, and
>>>>>>>>> never whenever memory_cache_enabled=off.
>>>>>>>>>
>>>>>>>>> I'm not sure how to debug this further.  Do you have any suggestions
>>>>>>>>> on anything that I can change with pgpool to better isolate what's
>>>>>>>>> going wrong?


More information about the pgpool-general mailing list