[pgpool-general: 1658] Re: Config file suggestion / request

[web at mr-paradox.net]

Without exported resourcses you're not really doing dynamic node allocation, 
you're just hardcoding the postgres instances. So you cannot programmatically
add a new node.

A typical aws/cloud implementation would have a new postgres
slave start as needed based on load and then spin down when not
needed. 

Upon startup of a node, you'll register the postgres instance as 
a virtual resource with puppet.

Puppet will then update pgpool's config and notify it to reload

Pgpool will see the new node and start sending traffic it's way.


I have all of this working with the exception of the dynamically 
updating pgpool's config file to put it in / remove it from rotation.



On Tue, Apr 23, 2013 at 01:29:19AM +0200, Maik Kulbe wrote:
- I can't say much about exported ressources, but I have managed to implement 
- a simple class.
- 
- You can take something in your template like
- 
- ===
- <% @backends.each_with_index do |server_ip,i| -%>
- ===
- 
- this assigns i a counter variable you can use for backend_hostname1, etc. 
- Then in your class file use
- 
- ===
- class pgpool (
-  $backends    = ['backend1', 'backend2'],
- ...
- ) {
- ...
- }
- ===
- 
- So you only have to initiate a class with some parameters for a host and 
- Puppet sets everything up. I hope that's enaugh to give you some hints on 
- how to create something to your needs - you might need more parameters. But 
- this way you have an easy way to create and manage a new pool server.
- 
- It probably would be the best to create a generalized class for this that 
- would be enaugh for most use cases.
- 
- >Yes, I tried going through the loops in a template, but
- >it didn't really work well with puppet's exported resources.
- >http://docs.puppetlabs.com/guides/exported_resources.html
- >
- >Specifically because when you instantiate the exported resources you
- >don't get an array, you just get your exported resource exeucted 'n'
- >times.
- >
- >In that case the loop counter would always be zero
- >
- >In your environment are you using exported resources? If so I'd love to
- >see
- >how you handle the  portion.
- >
- >
- >
- >On Mon, Apr 22, 2013 at 11:36:15PM +0200, Maik Kulbe wrote:
- >- I use Puppet myself for a small test cluster and I haven't had a huge
- >
- >- problem, even though I was new to puppet. You can make a template with
- >a
- >- loop and a counter variable that just echos this variable into the
- >sections
- >- in the config. That way you have just one block of template code. It's
- >
- >- fairly simple to implement, really.
- >-
- >- Just take a look at
- >- http://docs.puppetlabs.com/guides/templating.html#iteration - it is
- >just a
- >- simple ruby loop.
- >-
- >- I think it would be better to provide templates or classes for the
- >most
- >- populare management engines, like Puppet and Chef, than to make big
- >changes
- >- to how the config is handled and thus breaking public API consistency
- >
- >- throughout the versions.
- >-
- >- >Hello!
- >- >
- >- >I have a request for a change to the way the config files are
- >handled.
- >- >
- >- >Right now, the way the config file works it is very difficult to
- >- >dynamically add / remove nodes programmatically. (i.e., via
- >Puppet/Chef,
- >- >etc.)
- >- >
- >- >Which I think would be a huge boost to pgpool's popularity and
- >- >usefulness to
- >- >be able to do this.
- >- >I currently use it to load balance nodes in Amazon and it works fine,
- >if
- >- >i could
- >- >programmatically add and remove nodes then it would be perfect.
- >- >
- >- >The key problem is the way that each stanza is identified (the
- >- >sequential values
- >- >at the end of the parameter name) backend_hostname0,
- >backend_hostname1,
- >- >etc.
- >- >
- >- >That structure is very difficult to create via puppet.
- >- >
- >- >There are plenty of ways to solve this, but I think the current
- >- >standard
- >- >way would be the "conf.d/*" pattern. Where each node would have a
- >file
- >- >under
- >- >a conf.d/ directory and then the primary config file would import
- >all
- >- >files
- >- >in that directory. (similar to apache/nginx/etc)
- >- >
- >- >So under etc/conf.d/ would be (one for each host)
- >- >hostname1.conf
- >- >hostname2.conf
- >- >
- >- >and in there you'd just have one stanza of:
- >- >backend_hostname=''
- >- >backend_port=''
- >- >backend_weight=''
- >- >backend_data_directory=''
- >- >backend_flag=''
- >- >
- >- >I took a crack at the config file myself and didn't get that far.
- >(I'm
- >- >not fantastic with
- >- >C) =).
- >- >
- >- >Thanks
- >- >_______________________________________________
- >- >pgpool-general mailing list
- >- >pgpool-general at pgpool.net
- >- >http://www.pgpool.net/mailman/listinfo/pgpool-general
- >-
- >_______________________________________________
- >pgpool-general mailing list
- >pgpool-general at pgpool.net
- >http://www.pgpool.net/mailman/listinfo/pgpool-general
-