[Pgpool-hackers] patch "Caching query results in pgpool-II"

Thu Jun 16 00:37:37 UTC 2011

I fixed errors and these program can be compiled.

Some functions is declared in header file. Take memqcache.h for example.
Other c files include memqcache.h in order to use functions in memqcache.c.
(Refer to pool_process_context.c, pool_process_context.h)

[memqcache.h]
extern POOL_STATUS memqcache_lookup(POOL_
CONNECTION *frontend, POOL_CONNECTION_POOL *backend, Node *node);
extern int set_cache_on_memcached(POOL_CONNECTION *frontend, const char
*md5_query, const char *data);
extern int get_cache_on_memcached(POOL_CONNECTION *frontend, const char
*md5_query, char **buf, size_t *len);
extern int delete_cache_on_memcached(const char *key);
...and so on.

Pgpool.conf.sample is included.

In pool_memqcache.c, I removed header file which is not being used.

Static functions is declared before the function body is defined.

> memcached.h and related stuffs should not be included
> unconditionaly. I prefer to enable memcached in
> configure(i.e. configure --with-memcached=... style). Pgpool
> hackers, I would like to hear opinions for this though.

Now memcached.h is included in pool.h.

"#include <libmemcached/memcached.h>"

I implemented it without minding problem.
If need to take a method of doing "configure", I do so.

Now I am writing documents(doc/pgpool-en.html and doc/pgpool-ja.html).
It will take more time to complete it.

Thanks
Masanori YAMAZAKI

2011/6/15 Masanori Yamazaki <m.yamazaki23 at gmail.com>

> 2011/6/15 Tatsuo Ishii <ishii at sraoss.co.jp>
>
>> Some comments to this patch:
>>
>> - Static functions should be declared before the function body is
>> defined.
>>
> I correct it.
>
>
>> - What memcache library are you using? There are some and you do not
>>  explain which one you are using. Maybe this?
>>
>>  http://tangent.org/552/libmemcached.html
>>
>>
> Yes, memcached library I am using is "libmemcached".Sorry I forgot to
> explain it.
> http://libmemcached.org/libMemcached.html
>
>
>
>> - memcached.h and related stuffs should not be included
>>  unconditionaly. I prefer to enable memcached in
>>  configure(i.e. configure --with-memcached=... style). Pgpool
>>  hackers, I would like to hear opinions for this though.
>>
>>
> - No pgpool.conf.sample* included
>>
> I correct it
>
>
>> - No documents(doc/pgpool-en.html and doc/pgpool-ja.html)
>>  included. These should be provided in the first place.
>>
>
> I add what is implemented to these documents.
>
>
>
>> - Did not compile:
>>
>> gcc -DHAVE_CONFIG_H -DDEFAULT_CONFIGDIR=\"/usr/local/etc\" -I.
>>  -D_GNU_SOURCE -I /usr/local/pgsql/include   -g -O2 -Wall
>> -Wmissing-prototypes -Wmissing-declarations -MT child.o -MD -MP -MF
>> .deps/child.Tpo -c -o child.o child.c
>> child.c: In function 'do_child':
>> child.c:153: warning: implicit declaration of function
>> 'init_memcached_connection'
>> child.c: At top level:
>> child.c:1966: warning: conflicting types for 'init_memcached_connection'
>> child.c:1966: error: static declaration of 'init_memcached_connection'
>> follows non-static declaration
>> child.c:153: error: previous implicit declaration of
>> 'init_memcached_connection' was here
>> child.c: In function 'init_memcached_connection':
>> child.c:1969: warning: implicit declaration of function
>> 'memcached_connect'
>> child.c:1970: error: 'memcached_con' undeclared (first use in this
>> function)
>> child.c:1970: error: (Each undeclared identifier is reported only once
>> child.c:1970: error: for each function it appears in.)
>> make[2]: *** [child.o] Error 1
>> make[2]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
>> make[1]: *** [all-recursive] Error 1
>> make[1]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
>> make: *** [all] Error 2
>>
>
> I correct an error of program and send new patch file as soon as possible.
>
> comment thanks
>
>
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>> > 2011/6/14 Tatsuo Ishii <ishii at sraoss.co.jp>
>> >
>> >> Please explain what is implemented and what is not implemented in this
>> >> patch.
>> >>
>> >
>> > What is implemented in this patch are as below.
>> > - connect to memcached.
>> > - set RowDescription or DataRow on memcached.
>> > - get RowDescription or DataRow on memcached by using key of md5(SELECT
>> > convert).
>> > - delete cache on memcached by using key of md5(SELECT convert). only
>> > module.
>> > - fetch and parse the fellowing data in pgpool.conf.
>> >
>> > memory_cache_enabled
>> > memqcache_method
>> > memqcache_memcached_host
>> > memqcache_memcached_port
>> > memqcache_total_size
>> > memqcache_expire
>> > memqcache_maxcache
>> > memqcache_cache_block_size
>> >
>> > - add data to buffer before set data on memcached.
>> > - get buffer in order to set it on memcached
>> > - initialize buffer
>> >
>> > I am implemented only in Simple Query case.
>> >
>> >
>> > What is not implemented in this patch are as below.
>> > - shmem cache.
>> > structure of  managing table oids.
>> > the space management map.
>> > on memory query cache structure.
>> >
>> > - implementation in Extended Query case.
>> >
>> > - cache validation.
>> > The result of SELECT using none IMMUTABLE functions is not cached.
>> >
>> > - remove cache
>> > If cache is full, the least recently used cache is removed.
>> > If a table is dropped or modified, related cache data is deleted.
>> > If a table definition is modified(ALTER TABLE), related cache data is
>> > deleted.
>> > If a schema or database is dropped, related cache data is deleted.
>> >
>> >
>> > This week I am mainly implementing the shmem cache.
>> >
>> > Regards
>> > Masanori YAMAZAKI
>> >
>> >
>> >
>> >
>> >> --
>> >> Tatsuo Ishii
>> >> SRA OSS, Inc. Japan
>> >> English: http://www.sraoss.co.jp/index_en.php
>> >> Japanese: http://www.sraoss.co.jp
>> >>
>> >> > Hello, hackers
>> >> >
>> >> > I have implemented a function caching query on memcached.
>> >> > I am sending the patch file and it's so nice to give me some
>> opinions.
>> >> >
>> >> > Main program which I implemented is as below.
>> >> > - Search for cache data on memcached by using md5 which converted
>> SELECT
>> >> > into.
>> >> > - Set data on memcached. The key is md5 which converted  SELECT into.
>> >> > The value is RowDescription, DataRow from Backend.
>> >> >
>> >> > [Modify files]
>> >> > - pool.h
>> >> > - child.c
>> >> > - pool_config.l
>> >> > - pool_config.h
>> >> > - pool_config.c
>> >> > - pool_system.c
>> >> > - pool_proto_modules.c
>> >> > - pool_process_query.c
>> >> >
>> >> > [New File]
>> >> > - pool_memqcache.c
>> >> >
>> >> >
>> >> > --
>> >> > Regards
>> >> > Masanori YAMAZAKI
>> >>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://pgfoundry.org/pipermail/pgpool-hackers/attachments/20110616/42e77de4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch.tar.gz
Type: application/x-gzip
Size: 6729 bytes
Desc: not available
URL: <http://pgfoundry.org/pipermail/pgpool-hackers/attachments/20110616/42e77de4/attachment.bin>