Waiting for PostgreSQL 15 – Add support for MERGE SQL command

On 28th of March 2022, Alvaro Herrera committed patch:

Add support for MERGE SQL command
 
MERGE performs actions that modify rows in the target table using a
source table or query. MERGE provides a single SQL statement that can
conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise
require multiple PL statements.  For example,
 
MERGE INTO target AS t
USING source AS s
ON t.tid = s.sid
WHEN MATCHED AND t.balance > s.delta THEN
  UPDATE SET balance = t.balance - s.delta
WHEN MATCHED THEN
  DELETE
WHEN NOT MATCHED AND s.delta > 0 THEN
  INSERT VALUES (s.sid, s.delta)
WHEN NOT MATCHED THEN
  DO NOTHING;
 
MERGE works with regular tables, partitioned tables and inheritance
hierarchies, including column and row security enforcement, as well as
support for row and statement triggers and transition tables therein.
 
MERGE is optimized for OLTP and is parameterizable, though also useful
for large scale ETL/ELT. MERGE is not intended to be used in preference
to existing single SQL commands for INSERT, UPDATE or DELETE since there
is some overhead.  MERGE can be used from PL/pgSQL.
 
MERGE does not support targetting updatable views or foreign tables, and
RETURNING clauses are not allowed either.  These limitations are likely
fixable with sufficient effort.  Rewrite rules are also not supported,
but it's not clear that we'd want to support them.
 
Author: Pavan Deolasee <pavan.deolasee@gmail.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Amit Langote <amitlangote09@gmail.com>
Author: Simon Riggs <simon.riggs@enterprisedb.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions)
Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions)
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)
Reviewed-by: Japin Li <japinli@hotmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com
Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql

To say that it's huge would be an understatement. Discussion about merge/upsert are as long as Pg itself.

In 2015, for Pg 10, we got INSERT … ON CONFLICT DO ….

Then, in 2018, for Pg 11, we almost got MERGE. But there were problems, and the patch was reverted.

Now, four years later it seems it's time has come.

If you want to read all the details, please consult the docs. For now, let's see how the syntax looks in somewhat simple case.

$ CREATE TABLE test (
    id int8 generated always AS IDENTITY,
    username text NOT NULL UNIQUE,
    touch_count int4 NOT NULL DEFAULT 0,
    PRIMARY KEY (id)
);
CREATE TABLE

Now, let's write query that will insert, or update, depending on whether given user exists:

$ MERGE INTO test t
USING (VALUES ('depesz')) AS i(un)
ON t.username = i.un
WHEN matched THEN
    UPDATE SET touch_count = touch_count + 1
WHEN NOT matched THEN
    INSERT (username, touch_count) VALUES (i.un, 1);
MERGE 1

Now. I inserted use ‘depesz', and the table was clear before. So content of the table is…:

$ SELECT * FROM test;
 id | username | touch_count 
----+----------+-------------
  1 | depesz   |           1
(1 ROW)

Looks good. What will happen if I'd try this merge again?

$ MERGE INTO test t
USING (VALUES ('depesz')) AS i(un)
ON t.username = i.un
WHEN matched THEN
    UPDATE SET touch_count = touch_count + 1
WHEN NOT matched THEN
    INSERT (username, touch_count) VALUES (i.un, 1);
MERGE 1
 
$ SELECT * FROM test;
 id | username | touch_count 
----+----------+-------------
  1 | depesz   |           2
(1 ROW)

The cool part is that when WHEN clause in the merge can take more comparisons. And you can have more of them.

For example, let's assume that if some user would get “inserted" over 3 times, it will get deleted:

$ MERGE INTO test t
USING (VALUES ('depesz')) AS i(un)
ON t.username = i.un
WHEN matched AND touch_count < 3 THEN
    UPDATE SET touch_count = touch_count + 1
WHEN matched THEN
    DELETE
WHEN NOT matched THEN
    INSERT (username, touch_count) VALUES (i.un, 1);
MERGE 1
 
$ SELECT * FROM test;
 id | username | touch_count 
----+----------+-------------
  1 | depesz   |           3
(1 ROW)
 
$ MERGE INTO test t
USING (VALUES ('depesz')) AS i(un)
ON t.username = i.un
WHEN matched AND touch_count < 3 THEN
    UPDATE SET touch_count = touch_count + 1
WHEN matched THEN
    DELETE
WHEN NOT matched THEN
    INSERT (username, touch_count) VALUES (i.un, 1);
MERGE 1
 
$ SELECT * FROM test;
 id | username | touch_count 
----+----------+-------------
(0 ROWS)

First merge worked great, incrementing touch_count to 3. And 2nd, saw the touch_count already at 3, so instead it deleted the row!

There is one important missing feature (in my eyes) – lack of RETURNING.

But, we might get it in future. And even if not – it's still great thing to have.

If you're planning on using this feature I strongly suggest you read the docs as they talk about more stuff than me in here.

HUGE thanks to all involved. Amazing stuff.

3 thoughts on “Waiting for PostgreSQL 15 – Add support for MERGE SQL command”

  1. Is it possible to merge two tables in-place?
    Fore example: I have ITEMS and ITEMS_NEXT and I want to synchronize them (also DELETE missing rows in ITEMS) without TRUNCATE ITEMS.
    I came up with a solution but it requires two MERGE or MERGE + DELETE… WHERE NOT EXISTS…

  2. In the proposed syntax there’s no “WHEN NOT MATCHED BY SOURCE” clause, which is essential to handle data deleted in source tables, e.g. for ETL or other data warehouse-related tasks. Are there plans to support it?

  3. @sequel:

    Please note that I am not official spokeperson for Pg dev team. I’m just *a* blogger, that writes about stuff related to Pg. I have no idea what the plans are, or what is their status.

    For this, I guess that contacting devs (pgsql-hackers? pgsql-general?) is much better than writing to random blogger somewhere.

Comments are closed.