Comments on: getting multiple values from sequences

By: Helge

Helge — Fri, 21 Mar 2008 12:18:25 +0000

There is one more (little known) alternative to the generate_series() which was discussed in the matching IRC chat
It is INSERT with the RETURNING clause (apparently available since 8.2):

INSERT INTO master table VALUES ( … ), ( … ), ( …)
RETURNING pkey;

This returns the primary key assigned by the database for each inserted record. Which can then be used to construct subsequent INSERTs in belongs-to style tables (eg addresses of contacts).
(w/o the RETURNING feature you would need to do individual INSERTs to grab generated pkeys using CURRVAL)

Basic flow with RETURNING is:
a) build INSERT statement for master records
BEGIN;
b) INSERT master records, RETURNING ids
c) collect ids and build INSERT statements for child records [O(N)]
d) INSERT child records
COMMIT;

Instead of:
0) SELECT nextval FROM generate_series [O(N)]
a) build INSERT statement for master records with IDs
b) build INSERT statements for child records
BEGIN;
c) INSERT master records, INSERT child records
COMMIT;

Advantage over generate_series(): one less SQL statement send to server. Disadvantage: transaction (including locks on master) is kept open longer since step c) can take a while for large import sets. The generate_series() runs completely out of TX bounds.

Anyways, the multi_nextval() is awesome for large sets (O(1), out of TX). Thanks depesz!

By: przemek tomczyk

przemek tomczyk — Fri, 21 Mar 2008 09:39:01 +0000

nah, same flaw as in 1st comment

By: przemek tomczyk

przemek tomczyk — Fri, 21 Mar 2008 09:33:46 +0000

and what about “basic approach” modified not to send values from db to client/application:

select min(nextval(‘seq’)) from generate_series(1,1000);
?

By: depesz

depesz — Thu, 20 Mar 2008 13:47:00 +0000

@maniek:
no, it doesn’t. sequences are outside of transactions.

By: maniek

maniek — Thu, 20 Mar 2008 13:24:56 +0000

Actually i think (have not checked) that you don`t need the advisory locks in this situation – the alter statement does get an exclusive lock on the sequence till the end of transaction anyway

By: depesz

depesz — Thu, 20 Mar 2008 12:41:09 +0000

@Steve Haslam:
of course i can, but it gets complex as we insert to many tables.

anyway – yes, it’s a perfectly viable solution for import question.

By: Steve Haslam

Steve Haslam — Thu, 20 Mar 2008 12:34:19 +0000

Ah yes, I see what you mean. So maybe a “safe” compromise would be to copy the data to a temp table and then use insert to essentially join the data up with nextval() calls for each row… which presumably would only be faster than inserting them manually because of having less network turnarounds…

By: depesz

depesz — Thu, 20 Mar 2008 12:18:59 +0000

no, because there is no guarateee that generated numbers will be consecutive.

i.e. it might happen that you will get:
10,11,12,15,17,18,19,…

By: Steve Haslam

Steve Haslam — Thu, 20 Mar 2008 12:17:32 +0000

Couldn’t you do:

create temp table generated(id int);

insert into generated(id)
select nextval(‘seq’) from generate_series(1,1000);

select min(id) as first_id, max(id) as last_id from generated;

drop table generated;