[Pgpool-hackers] patch "Caching query results in pgpool-II"

Thu Jun 16 15:09:08 UTC 2011

Hello

I wrote documents about query cache (shmem or memcached).
I'm sending the following patch file.
Your comment would be appreciated.

- pgpool-ja.html
- pgpool-en.html

Regards
Masanori YAMAZAKI

2011/6/16 Masanori Yamazaki <m.yamazaki23 at gmail.com>

> I fixed errors and these program can be compiled.
>
> Some functions is declared in header file. Take memqcache.h for example.
> Other c files include memqcache.h in order to use functions in memqcache.c.
> (Refer to pool_process_context.c, pool_process_context.h)
>
> [memqcache.h]
> extern POOL_STATUS memqcache_lookup(POOL_
> CONNECTION *frontend, POOL_CONNECTION_POOL *backend, Node *node);
> extern int set_cache_on_memcached(POOL_CONNECTION *frontend, const char
> *md5_query, const char *data);
> extern int get_cache_on_memcached(POOL_CONNECTION *frontend, const char
> *md5_query, char **buf, size_t *len);
> extern int delete_cache_on_memcached(const char *key);
> ...and so on.
>
> Pgpool.conf.sample is included.
>
> In pool_memqcache.c, I removed header file which is not being used.
>
> Static functions is declared before the function body is defined.
>
>
> > memcached.h and related stuffs should not be included
> > unconditionaly. I prefer to enable memcached in
> > configure(i.e. configure --with-memcached=... style). Pgpool
> > hackers, I would like to hear opinions for this though.
>
> Now memcached.h is included in pool.h.
>
> "#include <libmemcached/memcached.h>"
>
> I implemented it without minding problem.
> If need to take a method of doing "configure", I do so.
>
> Now I am writing documents(doc/pgpool-en.html and doc/pgpool-ja.html).
> It will take more time to complete it.
>
>
> Thanks
> Masanori YAMAZAKI
>
>
>
>
> 2011/6/15 Masanori Yamazaki <m.yamazaki23 at gmail.com>
>
>> 2011/6/15 Tatsuo Ishii <ishii at sraoss.co.jp>
>>
>>> Some comments to this patch:
>>>
>>> - Static functions should be declared before the function body is
>>> defined.
>>>
>> I correct it.
>>
>>
>>> - What memcache library are you using? There are some and you do not
>>>  explain which one you are using. Maybe this?
>>>
>>>  http://tangent.org/552/libmemcached.html
>>>
>>>
>> Yes, memcached library I am using is "libmemcached".Sorry I forgot to
>> explain it.
>> http://libmemcached.org/libMemcached.html
>>
>>
>>
>>> - memcached.h and related stuffs should not be included
>>>  unconditionaly. I prefer to enable memcached in
>>>  configure(i.e. configure --with-memcached=... style). Pgpool
>>>  hackers, I would like to hear opinions for this though.
>>>
>>>
>> - No pgpool.conf.sample* included
>>>
>> I correct it
>>
>>
>>> - No documents(doc/pgpool-en.html and doc/pgpool-ja.html)
>>>  included. These should be provided in the first place.
>>>
>>
>> I add what is implemented to these documents.
>>
>>
>>
>>> - Did not compile:
>>>
>>> gcc -DHAVE_CONFIG_H -DDEFAULT_CONFIGDIR=\"/usr/local/etc\" -I.
>>>  -D_GNU_SOURCE -I /usr/local/pgsql/include   -g -O2 -Wall
>>> -Wmissing-prototypes -Wmissing-declarations -MT child.o -MD -MP -MF
>>> .deps/child.Tpo -c -o child.o child.c
>>> child.c: In function 'do_child':
>>> child.c:153: warning: implicit declaration of function
>>> 'init_memcached_connection'
>>> child.c: At top level:
>>> child.c:1966: warning: conflicting types for 'init_memcached_connection'
>>> child.c:1966: error: static declaration of 'init_memcached_connection'
>>> follows non-static declaration
>>> child.c:153: error: previous implicit declaration of
>>> 'init_memcached_connection' was here
>>> child.c: In function 'init_memcached_connection':
>>> child.c:1969: warning: implicit declaration of function
>>> 'memcached_connect'
>>> child.c:1970: error: 'memcached_con' undeclared (first use in this
>>> function)
>>> child.c:1970: error: (Each undeclared identifier is reported only once
>>> child.c:1970: error: for each function it appears in.)
>>> make[2]: *** [child.o] Error 1
>>> make[2]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
>>> make[1]: *** [all-recursive] Error 1
>>> make[1]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
>>> make: *** [all] Error 2
>>>
>>
>> I correct an error of program and send new patch file as soon as possible.
>>
>> comment thanks
>>
>>
>>> --
>>> Tatsuo Ishii
>>> SRA OSS, Inc. Japan
>>> English: http://www.sraoss.co.jp/index_en.php
>>> Japanese: http://www.sraoss.co.jp
>>>
>>> > 2011/6/14 Tatsuo Ishii <ishii at sraoss.co.jp>
>>> >
>>> >> Please explain what is implemented and what is not implemented in this
>>> >> patch.
>>> >>
>>> >
>>> > What is implemented in this patch are as below.
>>> > - connect to memcached.
>>> > - set RowDescription or DataRow on memcached.
>>> > - get RowDescription or DataRow on memcached by using key of md5(SELECT
>>> > convert).
>>> > - delete cache on memcached by using key of md5(SELECT convert). only
>>> > module.
>>> > - fetch and parse the fellowing data in pgpool.conf.
>>> >
>>> > memory_cache_enabled
>>> > memqcache_method
>>> > memqcache_memcached_host
>>> > memqcache_memcached_port
>>> > memqcache_total_size
>>> > memqcache_expire
>>> > memqcache_maxcache
>>> > memqcache_cache_block_size
>>> >
>>> > - add data to buffer before set data on memcached.
>>> > - get buffer in order to set it on memcached
>>> > - initialize buffer
>>> >
>>> > I am implemented only in Simple Query case.
>>> >
>>> >
>>> > What is not implemented in this patch are as below.
>>> > - shmem cache.
>>> > structure of  managing table oids.
>>> > the space management map.
>>> > on memory query cache structure.
>>> >
>>> > - implementation in Extended Query case.
>>> >
>>> > - cache validation.
>>> > The result of SELECT using none IMMUTABLE functions is not cached.
>>> >
>>> > - remove cache
>>> > If cache is full, the least recently used cache is removed.
>>> > If a table is dropped or modified, related cache data is deleted.
>>> > If a table definition is modified(ALTER TABLE), related cache data is
>>> > deleted.
>>> > If a schema or database is dropped, related cache data is deleted.
>>> >
>>> >
>>> > This week I am mainly implementing the shmem cache.
>>> >
>>> > Regards
>>> > Masanori YAMAZAKI
>>> >
>>> >
>>> >
>>> >
>>> >> --
>>> >> Tatsuo Ishii
>>> >> SRA OSS, Inc. Japan
>>> >> English: http://www.sraoss.co.jp/index_en.php
>>> >> Japanese: http://www.sraoss.co.jp
>>> >>
>>> >> > Hello, hackers
>>> >> >
>>> >> > I have implemented a function caching query on memcached.
>>> >> > I am sending the patch file and it's so nice to give me some
>>> opinions.
>>> >> >
>>> >> > Main program which I implemented is as below.
>>> >> > - Search for cache data on memcached by using md5 which converted
>>> SELECT
>>> >> > into.
>>> >> > - Set data on memcached. The key is md5 which converted  SELECT
>>> into.
>>> >> > The value is RowDescription, DataRow from Backend.
>>> >> >
>>> >> > [Modify files]
>>> >> > - pool.h
>>> >> > - child.c
>>> >> > - pool_config.l
>>> >> > - pool_config.h
>>> >> > - pool_config.c
>>> >> > - pool_system.c
>>> >> > - pool_proto_modules.c
>>> >> > - pool_process_query.c
>>> >> >
>>> >> > [New File]
>>> >> > - pool_memqcache.c
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Regards
>>> >> > Masanori YAMAZAKI
>>> >>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://pgfoundry.org/pipermail/pgpool-hackers/attachments/20110617/bfd1cc19/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: doc.patch
Type: text/x-patch
Size: 5794 bytes
Desc: not available
URL: <http://pgfoundry.org/pipermail/pgpool-hackers/attachments/20110617/bfd1cc19/attachment.bin>