July 31st, 2012 by depesz | Tags: , , , , , | 8 comments »
Did it help? If yes - maybe you can help me?

Some time ago I was looking (warning: post in polish) for someone to teach me proper parsing.

One of really great polish Perl programmers – Dozzie – reached out, and helped me. By the way – thanks a lot, Dozzie.

Based on what he taught me, I started writing module for parsing SQL queries. By that I mean proper parsing, with grammar (using Parse::Eyapp), and not set of regular expressions.

My parser is not ready. To say it lightly. Very lightly.

For now, it just knows how to parse the simplest queries like:

  • select 1;
  • select ‘a' as b;

I am working very slowly on it, so don't expect any usable version in any defined future. I will get there, eventually, but it is a project that I work on in my free time, after I finish everything else that I could work on in given moment.

This post is intended to announce that I'm working on it (so I will have kind of obligation to do it). And, if anyone is interested – I more than welcome all contributors/reviewers, and perhaps even critics 🙂

Final note – if you'll review the code, and want to comment on ugly list of regexps in Lexer – I know. It will be eventually replaced by one regular expression, but since it will be regular expression built by Regexp::Optimizer – it will not really be readable (though it will be faster than current approach).

  1. 8 comments

  2. Jul 31, 2012

    Might it be useful to crib patterns from the core parser?

  3. Jul 31, 2012

    @David:
    I do use core parser to check for stuff, but I don’t want to copy/paste things.

    Reason is very simple – this project, aside from its basic functionality serves one important purpose: makes me learn parsing. As such – I want to write it from scratch. Of course using, as a helper, other parsers – from Pg or SQLite.

    Besides – PostgreSQL SQL parser is supposedly very complicated to use because it uses a lot of internal data. I didn’t test it fully, but I expect it to be true.

  4. Aug 2, 2012

    I got quite excited about the Marpa parser – if I had any work related to parsing I would try it 🙂

    http://www.jeffreykegler.com/marpa

  5. Aug 2, 2012

    @zby:
    List of features looks interesting, but I already invested some time to learn Eyapp. Plus – it’s more standard tool, basing its work on yacc.

    Maybe in future …

  6. Sep 12, 2012

    To sum up answers to frequently asked question: “WHY DO YOU WRITE SQL PARSER?”:

    1. To learn how to write parsers
    2. To have *working* sql parser, which I can then use for reformatter/beautifier/tidy type of program
    3. To be able to write a tool that would point to common problems with queries (like: column without table alias in a query with joins, or join with conditions in where clause).

  7. Apr 14, 2015

    I’ve completed my own sql parser and formatter, please check it would be nice in everyday use. http://sites.google.com/site/sqlformatter/

  8. Apr 14, 2015

    Quick note to others:
    the link Mak posted links to zip file, which contains EXE. No idea what is inside, I don’t run randomly downloaded executables from the internet, don’t have windows, and generally don’t trust code without sources available.

  9. Apr 14, 2015

    I share ur concept of getting anything from the internet. I posted that package uder this subject because I had same needs as yours when I decided to write this prog and I found its usefull on daily basis. Its windows executable, maybe others would be interested. Thanks for not deleting this advertisement 🙂

Leave a comment