Tag: csv

Quick and dirty loading of CSV files

Back in September 2025, David Fetter asked on IRC, about a tool to quickly load CSV to database. One that would require minimal configuration, will try to figure out as much as possible on its own.

I thought that it would be great idea. Plus, I'm trying to learn more JavaScript / Node, so figured I can try.

Work and life intervened, and I couldn't get to it.

In the mean time I also read a lot of praise for some AI tools for quickly starting projects, or adding functionality, or learning new stuff. So, I decided to try.

Used Claude to make the basics, and then worked to add more tests and functionality.

Long story short, the tool is here.

It does what David wanted – works as simply as possible. Assuming you have your environment configured, you can just:

=$ pg_csv_loader some_file.csv

And it will figure out delimiter, quote character, column names, datatypes, and will load it to default database – which works just like normal psql

Tested it on couple different csv files, and it seems to work. It also handled PostgreSQL csv logs, which are often problematic as they use literal new line characters, which is causing problems for some parsers.

I don't claim that I wrote it, but I did extend tests, and datatype detection, so I guess I have some partial credit 🙂

Hope you'll find it useful.

Waiting for PostgreSQL 15 – Add HEADER support to COPY text format

On 28th of January 2022, Peter Eisentraut committed patch:

Add HEADER support to COPY text format 
 
The COPY CSV format supports the HEADER option to output a header
line.  This patch adds the same option to the default text format.  On
input, the HEADER option causes the first line to be skipped, same as
with CSV.
 
Author: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
Discussion: https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g33gpe1rcKQqe6wVQDrFA@mail.gmail.com

Continue reading Waiting for PostgreSQL 15 – Add HEADER support to COPY text format

Waiting for PostgreSQL 12 – Add CSV table output mode in psql.

On 26th of November 2018, Tom Lane committed patch:

Add CSV table output mode in psql.
 
"\pset format csv", or --csv, selects comma-separated values table format.
This is compliant with RFC 4180, except that we aren't too picky about
whether the record separator is LF or CRLF; also, the user may choose a
field separator other than comma.
 
This output format is directly compatible with the server's COPY CSV
format, and will also be useful as input to other programs.  It's
considerably safer for that purpose than the old recommendation to
use "unaligned" format, since the latter couldn't handle data
containing the field separator character.
 
Daniel Vérité, reviewed by Fabien Coelho and David Fetter, some
tweaking by me
 
Discussion: https://postgr.es/m/a8de371e-006f-4f92-ab72-2bbe3ee78f03@manitou-mail.org

Continue reading Waiting for PostgreSQL 12 – Add CSV table output mode in psql.