[Pgpool-hackers] patch "Caching query results in pgpool-II"

[Tatsuo Ishii]

Some comments to this patch:

- Static functions should be declared before the function body is defined.

- What memcache library are you using? There are some and you do not
  explain which one you are using. Maybe this?

  http://tangent.org/552/libmemcached.html

- memcached.h and related stuffs should not be included
  unconditionaly. I prefer to enable memcached in
  configure(i.e. configure --with-memcached=... style). Pgpool
  hackers, I would like to hear opinions for this though.

- No pgpool.conf.sample* included

- No documents(doc/pgpool-en.html and doc/pgpool-ja.html)
  included. These should be provided in the first place.

- Did not compile:

gcc -DHAVE_CONFIG_H -DDEFAULT_CONFIGDIR=\"/usr/local/etc\" -I.  -D_GNU_SOURCE -I /usr/local/pgsql/include   -g -O2 -Wall -Wmissing-prototypes -Wmissing-declarations -MT child.o -MD -MP -MF .deps/child.Tpo -c -o child.o child.c
child.c: In function 'do_child':
child.c:153: warning: implicit declaration of function 'init_memcached_connection'
child.c: At top level:
child.c:1966: warning: conflicting types for 'init_memcached_connection'
child.c:1966: error: static declaration of 'init_memcached_connection' follows non-static declaration
child.c:153: error: previous implicit declaration of 'init_memcached_connection' was here
child.c: In function 'init_memcached_connection':
child.c:1969: warning: implicit declaration of function 'memcached_connect'
child.c:1970: error: 'memcached_con' undeclared (first use in this function)
child.c:1970: error: (Each undeclared identifier is reported only once
child.c:1970: error: for each function it appears in.)
make[2]: *** [child.o] Error 1
make[2]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
make: *** [all] Error 2
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> 2011/6/14 Tatsuo Ishii <ishii at sraoss.co.jp>
> 
>> Please explain what is implemented and what is not implemented in this
>> patch.
>>
> 
> What is implemented in this patch are as below.
> - connect to memcached.
> - set RowDescription or DataRow on memcached.
> - get RowDescription or DataRow on memcached by using key of md5(SELECT
> convert).
> - delete cache on memcached by using key of md5(SELECT convert). only
> module.
> - fetch and parse the fellowing data in pgpool.conf.
> 
> memory_cache_enabled
> memqcache_method
> memqcache_memcached_host
> memqcache_memcached_port
> memqcache_total_size
> memqcache_expire
> memqcache_maxcache
> memqcache_cache_block_size
> 
> - add data to buffer before set data on memcached.
> - get buffer in order to set it on memcached
> - initialize buffer
> 
> I am implemented only in Simple Query case.
> 
> 
> What is not implemented in this patch are as below.
> - shmem cache.
> structure of  managing table oids.
> the space management map.
> on memory query cache structure.
> 
> - implementation in Extended Query case.
> 
> - cache validation.
> The result of SELECT using none IMMUTABLE functions is not cached.
> 
> - remove cache
> If cache is full, the least recently used cache is removed.
> If a table is dropped or modified, related cache data is deleted.
> If a table definition is modified(ALTER TABLE), related cache data is
> deleted.
> If a schema or database is dropped, related cache data is deleted.
> 
> 
> This week I am mainly implementing the shmem cache.
> 
> Regards
> Masanori YAMAZAKI
> 
> 
> 
> 
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>> > Hello, hackers
>> >
>> > I have implemented a function caching query on memcached.
>> > I am sending the patch file and it's so nice to give me some opinions.
>> >
>> > Main program which I implemented is as below.
>> > - Search for cache data on memcached by using md5 which converted SELECT
>> > into.
>> > - Set data on memcached. The key is md5 which converted  SELECT into.
>> > The value is RowDescription, DataRow from Backend.
>> >
>> > [Modify files]
>> > - pool.h
>> > - child.c
>> > - pool_config.l
>> > - pool_config.h
>> > - pool_config.c
>> > - pool_system.c
>> > - pool_proto_modules.c
>> > - pool_process_query.c
>> >
>> > [New File]
>> > - pool_memqcache.c
>> >
>> >
>> > --
>> > Regards
>> > Masanori YAMAZAKI
>>