[Pgpool-general] Pgpool I vs Pgpool II

[Glyn Astill]

Hi Toshihiro,

I can confirm that patch works and we no longer see the memory leak.

Thanks
Glyn



----- Original Message -----
> From: Toshihiro Kitagawa <kitagawa at sraoss.co.jp>
> To: Glyn Astill <glynastill at yahoo.co.uk>; takizo <paulooi at takizo.com>
> Cc: "pgpool-general at pgfoundry.org" <pgpool-general at pgfoundry.org>
> Sent: Thursday, 22 September 2011, 10:57
> Subject: Re: [Pgpool-general] Pgpool I vs Pgpool II
> 
> Hi Glyn, takizo,
> 
> I created a patch to fix a massive memory leak of raw mode.
> The patch which was posted before by Yoshiyuki seems correct.
> However, I think it's better to call pool_query_context_destroy()
> than free_parser().
> 
> Would you try an attached patch?
> 
> The patch is for V3.0_STABLE branch, but it's applicable to
> pgpool-II 3.0.x too.
> 
> Regards,
> 
> -- 
> Toshihiro Kitagawa
> SRA OSS, Inc. Japan
> 
> On Thu, 8 Sep 2011 20:14:41 +0900
> Toshihiro Kitagawa <kitagawa at sraoss.co.jp> wrote:
> 
>>  Hi Glyn, takizo,
>> 
>>  Sorry for delay.
>> 
>>  I fixed some memory leaks in pgpool-II 3.1 and V3_0_STABLE.
>>  But I guess memory leaks are remaining as you say.
>> 
>>  Surely, free_parser() which was commented out might be problem.
>>  However, I think simply uncommenting leads unexpected new problem.
>>  So we have to fix those carefully.
>> 
>>  I will investigate and fix memory problems which was posted to
>>  this ML from now on.
>> 
>>  Regards,
>> 
>>  -- 
>>  Toshihiro Kitagawa
>>  SRA OSS, Inc. Japan
>> 
>>  On Thu, 8 Sep 2011 16:22:50 +0800
>>  takizo <paulooi at takizo.com> wrote:
>> 
>>  > Glyn, 
>>  > 
>>  > Surely, I will try 2.3.3 :) 
>>  > Thanks a lot 
>>  > 
>>  > --
>>  > Paul Ooi 
>>  > 
>>  > 
>>  > 
>>  > On Sep 8, 2011, at 4:00 PM, Glyn Astill wrote:
>>  > 
>>  > > Hi takizo,
>>  > > 
>>  > > We are currently using Pgpool-II 2.3.3 and it has been really 
> stable for us.  Unless you're after a specific feature in Pgpool-II 3.x you 
> could perhaps try it?
>>  > > 
>>  > > So far I've had no luck with the patch posted by Yoshiyuki, 
> even with that patch there is still a massive memory leak in 3.0.4.  It would 
> appear that pool_proto_modules.c has lots commented out calls to free_parser(), 
> many more than the single case that Yoshiyuki uncommented with his patch, so 
> either there is a bit of a mess there that needs sorting out by someone who 
> knows the overall logic, or the issue is elsewhere.
>>  > > 
>>  > > Glyn
>>  > > 
>>  > > From: takizo <paulooi at takizo.com>
>>  > > To: Glyn Astill <glynastill at yahoo.co.uk>
>>  > > Cc: "pgpool-general at pgfoundry.org" 
> <pgpool-general at pgfoundry.org>
>>  > > Sent: Thursday, 8 September 2011, 1:48
>>  > > Subject: Re: [Pgpool-general] Pgpool I vs Pgpool II
>>  > > 
>>  > > Gyln, 
>>  > > 
>>  > > Thanks. I happened to read through the thread after posted the 
> mail. It seems like I have to stick with Pgpool I until the developer is free. 
>>  > > 
>>  > > --
>>  > > takizo
>>  > > 
>>  > > 
>>  > > 
>>  > > On Sep 7, 2011, at 10:53 PM, Glyn Astill wrote:
>>  > > 
>>  > >> Hi takizo,
>>  > >> 
>>  > >> Yes this bug has been seen by numerous people on the list, 
> Yoshiyuki Asaba has posted a patch, however there has been no response from the 
> main Pgpool II developers.
>>  > >> 
>>  > >> I did ask a few times, and also asked if we'd be better 
> testing 3.1, but alas still no response.  I can only assume the devs are busy or 
> otherwise indisposed.
>>  > >> 
>>  > >> See the thread below for the patch.
>>  > >> 
>>  > >> 
> http://www.mail-archive.com/pgpool-general@pgfoundry.org/msg03141.html
>>  > >> 
>>  > >> Glyn
>>  > >> 
>>  > >> From: takizo <paulooi at takizo.com>
>>  > >> To: pgpool-general at pgfoundry.org
>>  > >> Sent: Wednesday, 7 September 2011, 15:02
>>  > >> Subject: [Pgpool-general] Pgpool I vs Pgpool II
>>  > >> 
>>  > >> Hi all, 
>>  > >> 
>>  > >> Today I have upgraded Pgpool I to Pgpool II. And I had to 
> roll back to Pgpool I due to memory hunger issue. 
>>  > >> 
>>  > >> My server is configured with 
>>  > >> PostgreSQL 8.2, 
>>  > >> running on FreeBSD 8.2
>>  > >> Intel Xeon box with 4 CPUs with Quad Core, total of 16 cores
>>  > >> Memory with 24GB
>>  > >> 
>>  > >> * While running on Pgpool I
>>  > >> I do no have memory hunger issue.  On postgresql.conf, I have 
> max_connections=200 and shared_buffers=2GB. 
>>  > >> Most of the time, I have 8GB memory in used and 10GB Free and 
> some in cached. Everything run just fine and perfectly good. 
>>  > >> 
>>  > >> * I tried upgraded to Pgpool II 
>>  > >> Same config on postgresql.conf, when I started PgPool II 
> (3.0.4), my active memory started increasing and it didn't drop at all. 
>>  > >> I left the server running and after about 20 minutes, 20GB 
> memory is in used, ended up it used swap memory because running out of memory. 
>>  > >> 
>>  > >> I run a reload/restart on pgpool, I gained back 14GB of free 
> memory, and slowly it started to taken up those freed memory in active memory 
> after about 15-20 minutes. 
>>  > >> I tried to bring down max_connections and shared_buffers 
> value, and the memory doesn't stop taking up. It still slowly gaining all 
> the memory it wants. 
>>  > >> 
>>  > >> Has anyone having that problem as well? I hope I can find 
> some answer here :) 
>>  > >> Wanted to try out pgpool II performance but this problem is 
> the stopper for now. 
>>  > >> 
>>  > >> Hope to get feedback from gurus in the house.
>>  > >> 
>>  > >> Thanks! 
>>  > >> 
>>  > >> --
>>  > >> takizo
>>  > >> _______________________________________________
>>  > >> Pgpool-general mailing list
>>  > >> Pgpool-general at pgfoundry.org
>>  > >> http://pgfoundry.org/mailman/listinfo/pgpool-general
>>  > >> 
>>  > >> 
>>  > > 
>>  > > 
>>  > > 
>>  > 
>> 
>>  _______________________________________________
>>  Pgpool-general mailing list
>>  Pgpool-general at pgfoundry.org
>>  http://pgfoundry.org/mailman/listinfo/pgpool-general
>> 
>