[Pgpool-hackers] patch "Caching query results in pgpool-II"

[Masanori Yamazaki]

2011/6/17 Tatsuo Ishii <ishii at sraoss.co.jp>

> You need to mention that users need to build pgpool with
> --with-memcached=... if users want to use memcached based query
> cache.
>

I correct documents(pgpool-ja.html/pgpool-en.html) for memcached users.

comment thanks.
Masanori YAMAZAKI

--
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp
>
> > Hello
> >
> > I wrote documents about query cache (shmem or memcached).
> > I'm sending the following patch file.
> > Your comment would be appreciated.
> >
> > - pgpool-ja.html
> > - pgpool-en.html
> >
> > Regards
> > Masanori YAMAZAKI
> >
> >
> >
> >
> > 2011/6/16 Masanori Yamazaki <m.yamazaki23 at gmail.com>
> >
> >> I fixed errors and these program can be compiled.
> >>
> >> Some functions is declared in header file. Take memqcache.h for example.
> >> Other c files include memqcache.h in order to use functions in
> memqcache.c.
> >> (Refer to pool_process_context.c, pool_process_context.h)
> >>
> >> [memqcache.h]
> >> extern POOL_STATUS memqcache_lookup(POOL_
> >> CONNECTION *frontend, POOL_CONNECTION_POOL *backend, Node *node);
> >> extern int set_cache_on_memcached(POOL_CONNECTION *frontend, const char
> >> *md5_query, const char *data);
> >> extern int get_cache_on_memcached(POOL_CONNECTION *frontend, const char
> >> *md5_query, char **buf, size_t *len);
> >> extern int delete_cache_on_memcached(const char *key);
> >> ...and so on.
> >>
> >> Pgpool.conf.sample is included.
> >>
> >> In pool_memqcache.c, I removed header file which is not being used.
> >>
> >> Static functions is declared before the function body is defined.
> >>
> >>
> >> > memcached.h and related stuffs should not be included
> >> > unconditionaly. I prefer to enable memcached in
> >> > configure(i.e. configure --with-memcached=... style). Pgpool
> >> > hackers, I would like to hear opinions for this though.
> >>
> >> Now memcached.h is included in pool.h.
> >>
> >> "#include <libmemcached/memcached.h>"
> >>
> >> I implemented it without minding problem.
> >> If need to take a method of doing "configure", I do so.
> >>
> >> Now I am writing documents(doc/pgpool-en.html and doc/pgpool-ja.html).
> >> It will take more time to complete it.
> >>
> >>
> >> Thanks
> >> Masanori YAMAZAKI
> >>
> >>
> >>
> >>
> >> 2011/6/15 Masanori Yamazaki <m.yamazaki23 at gmail.com>
> >>
> >>> 2011/6/15 Tatsuo Ishii <ishii at sraoss.co.jp>
> >>>
> >>>> Some comments to this patch:
> >>>>
> >>>> - Static functions should be declared before the function body is
> >>>> defined.
> >>>>
> >>> I correct it.
> >>>
> >>>
> >>>> - What memcache library are you using? There are some and you do not
> >>>>  explain which one you are using. Maybe this?
> >>>>
> >>>>  http://tangent.org/552/libmemcached.html
> >>>>
> >>>>
> >>> Yes, memcached library I am using is "libmemcached".Sorry I forgot to
> >>> explain it.
> >>> http://libmemcached.org/libMemcached.html
> >>>
> >>>
> >>>
> >>>> - memcached.h and related stuffs should not be included
> >>>>  unconditionaly. I prefer to enable memcached in
> >>>>  configure(i.e. configure --with-memcached=... style). Pgpool
> >>>>  hackers, I would like to hear opinions for this though.
> >>>>
> >>>>
> >>> - No pgpool.conf.sample* included
> >>>>
> >>> I correct it
> >>>
> >>>
> >>>> - No documents(doc/pgpool-en.html and doc/pgpool-ja.html)
> >>>>  included. These should be provided in the first place.
> >>>>
> >>>
> >>> I add what is implemented to these documents.
> >>>
> >>>
> >>>
> >>>> - Did not compile:
> >>>>
> >>>> gcc -DHAVE_CONFIG_H -DDEFAULT_CONFIGDIR=\"/usr/local/etc\" -I.
> >>>>  -D_GNU_SOURCE -I /usr/local/pgsql/include   -g -O2 -Wall
> >>>> -Wmissing-prototypes -Wmissing-declarations -MT child.o -MD -MP -MF
> >>>> .deps/child.Tpo -c -o child.o child.c
> >>>> child.c: In function 'do_child':
> >>>> child.c:153: warning: implicit declaration of function
> >>>> 'init_memcached_connection'
> >>>> child.c: At top level:
> >>>> child.c:1966: warning: conflicting types for
> 'init_memcached_connection'
> >>>> child.c:1966: error: static declaration of 'init_memcached_connection'
> >>>> follows non-static declaration
> >>>> child.c:153: error: previous implicit declaration of
> >>>> 'init_memcached_connection' was here
> >>>> child.c: In function 'init_memcached_connection':
> >>>> child.c:1969: warning: implicit declaration of function
> >>>> 'memcached_connect'
> >>>> child.c:1970: error: 'memcached_con' undeclared (first use in this
> >>>> function)
> >>>> child.c:1970: error: (Each undeclared identifier is reported only once
> >>>> child.c:1970: error: for each function it appears in.)
> >>>> make[2]: *** [child.o] Error 1
> >>>> make[2]: Leaving directory
> `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
> >>>> make[1]: *** [all-recursive] Error 1
> >>>> make[1]: Leaving directory
> `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II'
> >>>> make: *** [all] Error 2
> >>>>
> >>>
> >>> I correct an error of program and send new patch file as soon as
> possible.
> >>>
> >>> comment thanks
> >>>
> >>>
> >>>> --
> >>>> Tatsuo Ishii
> >>>> SRA OSS, Inc. Japan
> >>>> English: http://www.sraoss.co.jp/index_en.php
> >>>> Japanese: http://www.sraoss.co.jp
> >>>>
> >>>> > 2011/6/14 Tatsuo Ishii <ishii at sraoss.co.jp>
> >>>> >
> >>>> >> Please explain what is implemented and what is not implemented in
> this
> >>>> >> patch.
> >>>> >>
> >>>> >
> >>>> > What is implemented in this patch are as below.
> >>>> > - connect to memcached.
> >>>> > - set RowDescription or DataRow on memcached.
> >>>> > - get RowDescription or DataRow on memcached by using key of
> md5(SELECT
> >>>> > convert).
> >>>> > - delete cache on memcached by using key of md5(SELECT convert).
> only
> >>>> > module.
> >>>> > - fetch and parse the fellowing data in pgpool.conf.
> >>>> >
> >>>> > memory_cache_enabled
> >>>> > memqcache_method
> >>>> > memqcache_memcached_host
> >>>> > memqcache_memcached_port
> >>>> > memqcache_total_size
> >>>> > memqcache_expire
> >>>> > memqcache_maxcache
> >>>> > memqcache_cache_block_size
> >>>> >
> >>>> > - add data to buffer before set data on memcached.
> >>>> > - get buffer in order to set it on memcached
> >>>> > - initialize buffer
> >>>> >
> >>>> > I am implemented only in Simple Query case.
> >>>> >
> >>>> >
> >>>> > What is not implemented in this patch are as below.
> >>>> > - shmem cache.
> >>>> > structure of  managing table oids.
> >>>> > the space management map.
> >>>> > on memory query cache structure.
> >>>> >
> >>>> > - implementation in Extended Query case.
> >>>> >
> >>>> > - cache validation.
> >>>> > The result of SELECT using none IMMUTABLE functions is not cached.
> >>>> >
> >>>> > - remove cache
> >>>> > If cache is full, the least recently used cache is removed.
> >>>> > If a table is dropped or modified, related cache data is deleted.
> >>>> > If a table definition is modified(ALTER TABLE), related cache data
> is
> >>>> > deleted.
> >>>> > If a schema or database is dropped, related cache data is deleted.
> >>>> >
> >>>> >
> >>>> > This week I am mainly implementing the shmem cache.
> >>>> >
> >>>> > Regards
> >>>> > Masanori YAMAZAKI
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> >> --
> >>>> >> Tatsuo Ishii
> >>>> >> SRA OSS, Inc. Japan
> >>>> >> English: http://www.sraoss.co.jp/index_en.php
> >>>> >> Japanese: http://www.sraoss.co.jp
> >>>> >>
> >>>> >> > Hello, hackers
> >>>> >> >
> >>>> >> > I have implemented a function caching query on memcached.
> >>>> >> > I am sending the patch file and it's so nice to give me some
> >>>> opinions.
> >>>> >> >
> >>>> >> > Main program which I implemented is as below.
> >>>> >> > - Search for cache data on memcached by using md5 which converted
> >>>> SELECT
> >>>> >> > into.
> >>>> >> > - Set data on memcached. The key is md5 which converted  SELECT
> >>>> into.
> >>>> >> > The value is RowDescription, DataRow from Backend.
> >>>> >> >
> >>>> >> > [Modify files]
> >>>> >> > - pool.h
> >>>> >> > - child.c
> >>>> >> > - pool_config.l
> >>>> >> > - pool_config.h
> >>>> >> > - pool_config.c
> >>>> >> > - pool_system.c
> >>>> >> > - pool_proto_modules.c
> >>>> >> > - pool_process_query.c
> >>>> >> >
> >>>> >> > [New File]
> >>>> >> > - pool_memqcache.c
> >>>> >> >
> >>>> >> >
> >>>> >> > --
> >>>> >> > Regards
> >>>> >> > Masanori YAMAZAKI
> >>>> >>
> >>>>
> >>>
> >>>
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://pgfoundry.org/pipermail/pgpool-hackers/attachments/20110618/5da4eeea/attachment.html>