There is a thread on pgsql-hackers mailing list, dating back to 1st of March 2020 about changes to Pg to optimize handling of larger number of connections.
The thread is pretty long, and the discussion took 5 months, but lately we got couple of commits:
- snapshot scalability: Don't compute global horizons while building snapshots.
- snapshot scalability: Move PGXACT->xmin back to PGPROC.
- snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags.
- snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.
- snapshot scalability: Introduce dense array of in-progress xids.
- snapshot scalability: cache snapshots using a xact completion counter.
- Fix race condition in snapshot caching when 2PC is used.
that (supposedly) should improve handling of large numbers of concurrent connections.
Unfortunately, I don't have ready test servers that would allow me to sensibly tests thousands of connections, but I think we can all safely assume that PostgreSQL 14 should handle large connection counts better than any PostgreSQL before. Thanks a lot, to all involved, but specifically to mastermind of this project: Andres Freund.
On 22nd of March, Tom Lane committed patch:
Allow foreign tables to participate in inheritance.
Foreign tables can now be inheritance children, or parents. Much of the
system was already ready for this, but we had to fix a few things of
course, mostly in the area of planner and executor handling of row locks.
As side effects of this, allow foreign tables to have NOT VALID CHECK
constraints (and hence to accept ALTER ... VALIDATE CONSTRAINT), and to
accept ALTER SET STORAGE and ALTER SET WITH/WITHOUT OIDS. Continuing to
disallow these things would've required bizarre and inconsistent special
cases in inheritance behavior. Since foreign tables don't enforce CHECK
constraints anyway, a NOT VALID one is a complete no-op, but that doesn't
mean we shouldn't allow it. And it's possible that some FDWs might have
use for SET STORAGE or SET WITH OIDS, though doubtless they will be no-ops
An additional change in support of this is that when a ModifyTable node
has multiple target tables, they will all now be explicitly identified
in EXPLAIN output, for example:
Update on pt1 (cost=0.00..321.05 rows=3541 width=46)
Update on pt1
Foreign Update on ft1
Foreign Update on ft2
Update on child3
-> Seq Scan on pt1 (cost=0.00..0.00 rows=1 width=46)
-> Foreign Scan on ft1 (cost=100.00..148.03 rows=1170 width=46)
-> Foreign Scan on ft2 (cost=100.00..148.03 rows=1170 width=46)
-> Seq Scan on child3 (cost=0.00..25.00 rows=1200 width=46)
This was done mainly to provide an unambiguous place to attach "Remote SQL"
fields, but it is useful for inherited updates even when no foreign tables
Shigeru Hanada and Etsuro Fujita, reviewed by Ashutosh Bapat and Kyotaro
Horiguchi, some additional hacking by me
Continue reading Waiting for 9.5 – Allow foreign tables to participate in inheritance. – A.K.A. PostgreSQL got sharding.
( title, in case you don't know, comes from excellent “movie" about databases )
The secret ingredient is of course sharding.
Can we do sharding in PostgreSQL? First, let's define what sharding is:
Continue reading The secret ingredient in the webscale sauce
as you perhaps know there is this site/blog called high scalability. it contains articles about various things related to performance, scalability, availability and so on.
usually, when there is something about databases, it is about mysql. luckily for us, today they showed something about postgresql:
“Skype Plans for PostgreSQL to Scale to 1 Billion Users“.