Waiting for PostgreSQL 19 – Online enabling and disabling of data checksums

On 3rd of April 2026, Daniel Gustafsson committed patch:

Online enabling and disabling of data checksums
 
This allows data checksums to be enabled, or disabled, in a running
cluster without restricting access to the cluster during processing.
 
Data checksums could prior to this only be enabled during initdb or
when the cluster is offline using the pg_checksums app. This commit
introduce functionality to enable, or disable, data checksums while
the cluster is running regardless of how it was initialized.
 
A background worker launcher process is responsible for launching a
dynamic per-database background worker which will mark all buffers
dirty for all relation with storage in order for them to have data
checksums calculated on write.  Once all relations in all databases
have been processed, the data_checksums state will be set to on and
the cluster will at that point be identical to one which had data
checksums enabled during initialization or via offline processing.
 
When data checksums are being enabled, concurrent I/O operations
from backends other than the data checksums worker will write the
checksums but not verify them on reading.  Only when all backends
have absorbed the procsignalbarrier for setting data_checksums to
on will they also start verifying checksums on reading.  The same
process is repeated during disabling; all backends write checksums
but do not verify them until the barrier for setting the state to
off has been absorbed by all.  This in-progress state is used to
ensure there are no false negatives (or positives) due to reading
a checksum which is not in sync with the page.
 
A new testmodule, test_checksums, is introduced with an extensive
set of tests covering both online and offline data checksum mode
changes.  The tests which run concurrent pgbdench during online
processing are gated behind the PG_TEST_EXTRA flag due to being
very expensive to run.  Two levels of PG_TEST_EXTRA flags exist
to turn on a subset of the expensive tests, or the full suite of
multiple runs.
 
This work is based on an earlier version of this patch which was
reviewed by among others Heikki Linnakangas, Robert Haas, Andres
Freund, Tomas Vondra, Michael Banck and Andrey Borodin.  During
the work on this new version, Tomas Vondra has given invaluable
assistance with not only coding and reviewing but very in-depth
testing.
 
Author: Daniel Gustafsson <daniel@yesql.se>
Author: Magnus Hagander <magnus@hagander.net>
Co-authored-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CABUevExz9hUUOLnJVr2kpw9Cx=o4MCr1SVKwbupzuxP7ckNutA@mail.gmail.com
Discussion: https://postgr.es/m/20181030051643.elbxjww5jjgnjaxg@alap3.anarazel.de
Discussion: https://postgr.es/m/CABUevEwE3urLtwxxqdgd5O2oQz9J717ZzMbh+ziCSa5YLLU_BA@mail.gmail.com

WOW!

Back in 2013, in version 9.3, we got data checksums. Basically a way for PostgreSQL to detect data corruption, and inform dba about it by failing early.

Great. The thing is that it changes data on disk, so it needed to be enabled while running initdb, and couldn't have been changed.

Since then most all prepackaged Pg installations switched to enabling checksums by default, and from PostgreSQL 18 they even get enabled by default straight from PostgreSQL – you have to actively use –no-data-checksums option if, for whatever reason, you don't want them.

But if you are working on Pg that was created long time ago, or someone didn't enable checksums, you could have pg_upgraded to newest version, but checksums were still disabled.

Now, this will no longer be true.

Let's see how that works. First, let's have Pg with data checksums disabled:

=$ initdb --no-data-checksums --no-instructions -A trust -c port=5666 tmp/pg/
The files belonging to this database system will be owned by user "depesz".
This user must also own the server process.
 
The database cluster will be initialized with this locale configuration:
  locale provider:   libc
  LC_COLLATE:  en_US.UTF-8
  LC_CTYPE:    en_US.UTF-8
  LC_MESSAGES: en_US.UTF-8
  LC_MONETARY: en_US.UTF-8
  LC_NUMERIC:  en_US.UTF-8
  LC_TIME:     en_GB.UTF-8
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
 
Data page checksums are disabled.
 
creating directory tmp/pg ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default "max_connections" ... 100
selecting default "shared_buffers" ... 128MB
selecting default time zone ... Europe/Warsaw
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Sweet, now, let's start it, and see the status:

=$ pg_ctl -l tmp/pg.log  -D tmp/pg start
waiting for server to start.... done
server started
 
=$ psql -p 5666 -d postgres
psql (19devel)
Type "help" for help.
 
(depesz@[local]:5666) 17:42:24 [postgres]
$ show data_checksums ;
 data_checksums
────────────────
 off
(1 row)

Great. Now, disabling and enabling is trivial:

=$ select pg_enable_data_checksums();
 pg_enable_data_checksums
──────────────────────────
 
(1 row)
 
=$ show data_checksums ;
 data_checksums
────────────────
 on
(1 row)
 
=$ select pg_disable_data_checksums();
 pg_disable_data_checksums
───────────────────────────
 
(1 row)
 
=$ show data_checksums ;
 data_checksums
────────────────
 off
(1 row)

The thing is that both enabling and disabling require rewrite of all pages on disk for given cluster. So it can take non-trivial time.

Let's see how that would work. To see it, in one terminal, I ran:

=$ select format('%s : checksums: %s', now()::timestamp(0), current_setting('data_checksums')) \watch 1
2026-04-15 19:47:39 : checksums: off
2026-04-15 19:47:40 : checksums: off
2026-04-15 19:47:41 : checksums: off
2026-04-15 19:47:42 : checksums: off
…

And I let it ran.

In the first window I made some data:

=$ create table whatever as
    select i, repeat('x'||i::text, cast(10 + random() * 10 as int4)) as payload
    from generate_series(1,10000000) i;

This made me a 1.4GB table, in ~ 9 seconds.

So, let's see the enabling that is async…

=$ select now(); show data_checksums; select pg_enable_data_checksums(); select now();
              now
───────────────────────────────
 2026-04-15 19:51:59.568094+02
(1 row)
 
Time: 0.433 ms
 data_checksums
────────────────
 off
(1 row)
 
Time: 0.570 ms
 pg_enable_data_checksums
──────────────────────────
 
(1 row)
 
Time: 0.255 ms
              now
───────────────────────────────
 2026-04-15 19:51:59.584742+02
(1 row)
 
Time: 0.265 ms

As you can see enabling finished run at 19:51:59.584…, and it took 0.255ms.

In the mean time, the window that was showing me state of data_checksums showed:

2026-04-15 19:51:57 : checksums: off
2026-04-15 19:51:58 : checksums: off
2026-04-15 19:51:59 : checksums: off
2026-04-15 19:52:00 : checksums: inprogress-on
2026-04-15 19:52:01 : checksums: inprogress-on
2026-04-15 19:52:02 : checksums: inprogress-on
2026-04-15 19:52:03 : checksums: on
2026-04-15 19:52:04 : checksums: on
2026-04-15 19:52:05 : checksums: on

What about disabling?

=$ select now(); show data_checksums; select pg_disable_data_checksums(); select now();
              now
───────────────────────────────
 2026-04-15 19:53:19.202801+02
(1 row)
 
Time: 0.424 ms
 data_checksums
────────────────
 on
(1 row)
 
Time: 0.225 ms
 pg_disable_data_checksums
───────────────────────────
 
(1 row)
 
Time: 0.447 ms
              now
───────────────────────────────
 2026-04-15 19:53:19.218683+02
(1 row)
 
Time: 0.471 ms

and state showed:

2026-04-15 19:53:15 : checksums: on
2026-04-15 19:53:16 : checksums: on
2026-04-15 19:53:17 : checksums: on
2026-04-15 19:53:18 : checksums: on
2026-04-15 19:53:19 : checksums: off
2026-04-15 19:53:20 : checksums: off
2026-04-15 19:53:21 : checksums: off
2026-04-15 19:53:22 : checksums: off

This time there wasn't any “inprogress". It was just straight to off. It probably would work a bit differently under high load, but the gist is that while the command to enable/disable is instantaneous, there is some (or a lot, depending on how you look at it) background work to be done by rewriting every single table and index in the system.

This rewrite can be pretty IO intensive, and because of this pg_enable_data_checksums() function has two optional arguments:

=$ \df pg_enable_data_checksums
                                                       List of functions
   Schema   │           Name           │ Result data type │                     Argument data types                      │ Type
────────────┼──────────────────────────┼──────────────────┼──────────────────────────────────────────────────────────────┼──────
 pg_catalog │ pg_enable_data_checksums │ void             │ cost_delay integer DEFAULT 0, cost_limit integer DEFAULT 100 │ func
(1 row)

cost_delay is number of milliseconds that background worker has to sleep once it reaches accumulated cost of cost_limit, using the same semantics as vacuum_cost_delay and vacuum_cost_limit.

Since the process requires “dirtying" every page, and default vacuum_cost_dirty is 20, it means that if I'd run:

=$ select pg_enable_data_checksums(1);

the backgrouns rewriter will sleep for 1ms every 5 pages (default limit is 100, and cost per page is 20, so 5 pages). 5 pages is 40kB. So this is 1ms sleep every 40kB of rewrite. Fully configurable.

Very, very nice. There are a lot of people that are on old, or old-but-upgraded Pg instances, that don't have checksums enabled, as this would require full re-initdb, and re-load of data. And now they can just enable it and be a bit safer.

Thanks a lot to everyone involved. Great work.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.