Using JSON: json vs. jsonb, pglz vs. lz4, key optimization, parsing speed?

Recently(ish) I had a conversation on one of PostgreSQL support chats (IRC, Slack, or Discord) about efficient storage of JSON data, which compression to use, which datatype.

Unrelated to this, some people (at least two over the last year or so) said that they aren't sure if PostgreSQL doesn't optimize storage between columns, for example, storing attribute names once per column, and not once per value.

Decided to investigate…

Continue reading Using JSON: json vs. jsonb, pglz vs. lz4, key optimization, parsing speed?

Fixes on why-upgrade.depesz.com

In case you're not familiar with this site, why-upgrade.depesz.com shows you aggregated changelog between any two releases, with optionally searching for some keywords.

Yesterday azeem on irc pointed me towards a problem on why-upgrade.depesz.com. Specifically, when displaying changes from 9.5.24 to 13.1 site showed 30 security fixes. But in reality there should be only 10.

Continue reading Fixes on why-upgrade.depesz.com

Fix for displaying aggregates on explain.depesz.com

Couple of days ago RhodiumToad reported, on irc, a bug in explain.depesz.com.

Specifically – if explain was done using JSON/XML/YAML formats, and node type was Aggregate, the site didn't extract full info.

Continue reading Fix for displaying aggregates on explain.depesz.com

Migrating simple table to partitioned. How?

Recently someone asked, on irc, how to make table partitioned.

The thing is that it was supposed to be done with new partitioning, and not the old way.

The problem is that while we can create table that will be seen as partitioned – we can't alter table to become partitioned.

So. Is it possible?

Continue reading Migrating simple table to partitioned. How?

Converting list of integers into list of ranges

Yesterday someone on irc asked:

i've a query that returns sequential numbers with gaps (generate_series + join) and my question is: can is somehow construct ranges out of the returned values? sort of range_agg or something?

There was no further discussion, aside from me saying

sure you can. not trivial task, but possible.
you'd need window functions.

but it got me thinking …

Continue reading Converting list of integers into list of ranges

Waiting for PostgreSQL 10 – Implement syntax for transition tables in AFTER triggers.

Another one missed, quite a long time ago, too..:

On 4th of November 2016, Kevin Grittner committed patch:

Implement syntax for transition tables in AFTER triggers.
 
 
This is infrastructure for the complete SQL standard feature.  No
support is included at this point for execution nodes or PLs.  The
intent is to add that soon.
 
As this patch leaves things, standard syntax can create tuplestores
to contain old and/or new versions of rows affected by a statement.
References to these tuplestores are in the TriggerData structure.
C triggers can access the tuplestores directly, so they are usable,
but they cannot yet be referenced within a SQL statement.

Continue reading Waiting for PostgreSQL 10 – Implement syntax for transition tables in AFTER triggers.

Getting list of unique elements in table, per group

Today, on irc, someone asked interesting question.

Basically she ran a query like:

select a, b, c, d, e, f from table order by a

then, she processed the query to get, for each a array of unique values of b, c, d, e, and f, and then he inserted it back to database, to some other table.

It was a problem, because the table had many rows (millions I would assume), and the whole process was slow.

So, how to make it faster?

Continue reading Getting list of unique elements in table, per group