[Pgpool-hackers] patch "Caching query results in pgpool-II"

[Tatsuo Ishii]

> 2011/6/14 Tatsuo Ishii <ishii at sraoss.co.jp>
> 
>> Please explain what is implemented and what is not implemented in this
>> patch.
>>
> 
> What is implemented in this patch are as below.
> - connect to memcached.
> - set RowDescription or DataRow on memcached.
> - get RowDescription or DataRow on memcached by using key of md5(SELECT
> convert).
> - delete cache on memcached by using key of md5(SELECT convert). only
> module.
> - fetch and parse the fellowing data in pgpool.conf.
> 
> memory_cache_enabled
> memqcache_method
> memqcache_memcached_host
> memqcache_memcached_port
> memqcache_total_size
> memqcache_expire
> memqcache_maxcache
> memqcache_cache_block_size
> 
> - add data to buffer before set data on memcached.
> - get buffer in order to set it on memcached
> - initialize buffer
> 
> I am implemented only in Simple Query case.
> 
> 
> What is not implemented in this patch are as below.
> - shmem cache.
> structure of  managing table oids.
> the space management map.
> on memory query cache structure.
> 
> - implementation in Extended Query case.
> 
> - cache validation.
> The result of SELECT using none IMMUTABLE functions is not cached.

I have made a patch to implement a function which checks if a SELECT
statement includes non IMMUTABLE function call.

extern bool pool_has_non_immutable_function_call(Node *node);

See attached patches.

TODO:

Some statements including:

SELECT current_timestamp;

is not detected by the function even if it uses IMMUTABLE function
because it is translated to a non function form. Obviously we should
not cache the result of current_timestamp. What should we do? Probably
we have to have a heuristics to detect the particular form?

> - remove cache
> If cache is full, the least recently used cache is removed.
> If a table is dropped or modified, related cache data is deleted.
> If a table definition is modified(ALTER TABLE), related cache data is
> deleted.
> If a schema or database is dropped, related cache data is deleted.
> 
> 
> This week I am mainly implementing the shmem cache.
> 
> Regards
> Masanori YAMAZAKI
> 
> 
> 
> 
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>> > Hello, hackers
>> >
>> > I have implemented a function caching query on memcached.
>> > I am sending the patch file and it's so nice to give me some opinions.
>> >
>> > Main program which I implemented is as below.
>> > - Search for cache data on memcached by using md5 which converted SELECT
>> > into.
>> > - Set data on memcached. The key is md5 which converted  SELECT into.
>> > The value is RowDescription, DataRow from Backend.
>> >
>> > [Modify files]
>> > - pool.h
>> > - child.c
>> > - pool_config.l
>> > - pool_config.h
>> > - pool_config.c
>> > - pool_system.c
>> > - pool_proto_modules.c
>> > - pool_process_query.c
>> >
>> > [New File]
>> > - pool_memqcache.c
>> >
>> >
>> > --
>> > Regards
>> > Masanori YAMAZAKI
>>