[Pgpool-hackers] Partial replication

[Guillaume Lelarge]

Le 21/02/2011 01:53, Tatsuo Ishii a écrit :
>> Le 18/02/2011 02:21, Tatsuo Ishii a écrit :
>>> Hi hackes,
>>>
>>> After my talk of pgpool-II V3 at FOSDEM 2011, I got a question about
>>> the idea "partial replication", which allows to replicate part of
>>> a database cluster in replication mode.
>>>
>>> This has been on my personal TODO list for years, and I start to think
>>> about the implementation design again.
>>>
>>> - We could have "database level partial replication" and "table level
>>>   partial replication".
>>>
>>> - "Database level partial replication" is easy to do. Just having a
>>>   list of databases to replicate (or not to replicate) and
>>>   pool_where_to_send() will look it. However cross database tables
>>>   such as pg_database must be always replicated.
>>>
>>> - "Table level partial replication" is not so easy. In addition to
>>>   having a list of tables to replicate (or not to replicate), we would
>>>   want to have such a list for schemas.
>>>
>>> - If we have such a schema list, we need to know which schema the
>>>   table belogs to. This is not super easy if the table is not schema
>>>   qualified but doable.
>>>
>>> - Functions would be troublesome because we cannot know which table it
>>>   is going to write.
>>>
>>> Just my memorandum. Please feel free to add comments.
>>
>> As you say, databases would be easy.
>>
>> I don't see an easy way to configure table names. Using a text file
>> would be a burden if the user has a big number of tables. If you want to
>> have such a feature, we need to provide a way to easyly configure the
>> replication feature. We already have a system DB available. We could
>> probably use it to "register" the tables to replicate.
> 
> Year to store large configuration data, we could use PostgreSQL
> itself. One thing I want to do is, getting ride of system DB. We could
> use one of DB nodes for this purpose, no?
> 

We need to have this on each node. It would be really sad if the only
node with the information just crash.

> Another problem we have to resolve is, online recovery. Probably we
> need to wait for the day when PostgreSQL can do the base backup
> partialy.

Meaning we'll wait quite a long time. Partial replication is a long way
to go.


-- 
Guillaume
 http://www.postgresql.fr
 http://dalibo.com