On 28th of June, Robert Haas committed patch:
Dramatically reduce System V shared memory consumption. Except when compiling with EXEC_BACKEND, we'll now allocate only a tiny amount of System V shared memory (as an interlock to protect the data directory) and allocate the rest as anonymous shared memory via mmap. This will hopefully spare most users the hassle of adjusting operating system parameters before being able to start PostgreSQL with a reasonable value for shared_buffers. There are a bunch of documentation updates needed here, and we might need to adjust some of the HINT messages related to shared memory as well. But it's not 100% clear how portable this is, so before we write the documentation, let's give it a spin on the buildfarm and see what turns red.
This patch doesn't add any new functionality, but removes one thing that had caused some issues.
As you perhaps know, PostgreSQL has so called “shared_buffers". In there, it stores various data, most importantly copies of data pages (8kB blocks).
Problem with shared_buffers is that you usually start by setting them to something like 20%-25% of available RAM, which with current multi-gigabyte servers is a non-trivial amount.
And most of the systems I've seen have very conservative limits on how much shared memory there can be. For example – my desktop Ubuntu 12.04 has the limit set to:
=$ cat /proc/sys/kernel/shmmax 33554432
This means that when you configure your PostgreSQL to actually use the memory it has for good use – i.e. for shared_buffers – you have to configure your kernel too.
And if you forget, or something fails to re-configure it on reboot – PostgreSQL will not start, showing errors like:
2012-07-12 12:25:13 CEST  : [1-1] user=,db=,e=XX000: FATAL: could not create shared memory segment: Invalid argument 2012-07-12 12:25:13 CEST  : [2-1] user=,db=,e=XX000: DETAIL: Failed system call was shmget(key=5910001, size=3318874112, 03600). 2012-07-12 12:25:13 CEST  : [3-1] user=,db=,e=XX000: HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter. You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 3318874112 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections. If the request size is already small, it's possible that it is less than your kernel's SHMMIN parameter, in which case raising the request size or reconfiguring SHMMIN is called for. The PostgreSQL documentation contains more information about shared memory configuration.
Error message is pretty helpful, so fixing it is usually not a problem. But why have the error in the first place, when you can skip it altogether?
Roberts commit does exactly it. Instead of using so called “System V shared memory" (which is the subject to limitation in SHMMAX), it switches to use shared memory by mmap.
Thanks to this, on the same machine, with the same 32MB limit for SHMMAX, I can start PostgreSQL 9.3, with shared_buffers = 3GB, and it works:
=$ ps -u pgdba f u USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND pgdba 7993 0.0 0.7 3268388 86136 ? S 12:28 0:00 /home/pgdba/work/bin/postgres pgdba 7997 0.0 0.0 24792 560 ? Ss 12:28 0:00 \_ postgres: logger process pgdba 7999 0.0 0.0 3269928 984 ? Ss 12:28 0:00 \_ postgres: checkpointer process pgdba 8000 0.0 0.1 3269928 15772 ? Ss 12:28 0:00 \_ postgres: writer process pgdba 8001 0.0 0.0 3269928 984 ? Ss 12:28 0:00 \_ postgres: wal writer process pgdba 8002 0.0 0.0 3270932 2456 ? Ss 12:28 0:00 \_ postgres: autovacuum launcher process pgdba 8003 0.0 0.0 26888 632 ? Ss 12:28 0:00 \_ postgres: archiver process pgdba 8004 0.0 0.0 27184 1240 ? Ss 12:28 0:00 \_ postgres: stats collector process
One less thing to worry about, and one less reason why starting Pg might fail. Of course it can still fail, if you'll configure shared_buffers larger than your actual memory size, but thats much less likely.