Some time ago I was looking (warning: post in polish) for someone to teach me proper parsing.
One of really great polish Perl programmers – Dozzie – reached out, and helped me. By the way – thanks a lot, Dozzie.
Based on what he taught me, I started writing module for parsing SQL queries. By that I mean proper parsing, with grammar (using Parse::Eyapp), and not set of regular expressions.
My parser is not ready. To say it lightly. Very lightly.
For now, it just knows how to parse the simplest queries like:
- select 1;
- select ‘a' as b;
I am working very slowly on it, so don't expect any usable version in any defined future. I will get there, eventually, but it is a project that I work on in my free time, after I finish everything else that I could work on in given moment.
This post is intended to announce that I'm working on it (so I will have kind of obligation to do it). And, if anyone is interested – I more than welcome all contributors/reviewers, and perhaps even critics 🙂
Final note – if you'll review the code, and want to comment on ugly list of regexps in Lexer – I know. It will be eventually replaced by one regular expression, but since it will be regular expression built by Regexp::Optimizer – it will not really be readable (though it will be faster than current approach).
Might it be useful to crib patterns from the core parser?
@David:
I do use core parser to check for stuff, but I don’t want to copy/paste things.
Reason is very simple – this project, aside from its basic functionality serves one important purpose: makes me learn parsing. As such – I want to write it from scratch. Of course using, as a helper, other parsers – from Pg or SQLite.
Besides – PostgreSQL SQL parser is supposedly very complicated to use because it uses a lot of internal data. I didn’t test it fully, but I expect it to be true.
I got quite excited about the Marpa parser – if I had any work related to parsing I would try it 🙂
http://www.jeffreykegler.com/marpa
@zby:
List of features looks interesting, but I already invested some time to learn Eyapp. Plus – it’s more standard tool, basing its work on yacc.
Maybe in future …
To sum up answers to frequently asked question: “WHY DO YOU WRITE SQL PARSER?”:
1. To learn how to write parsers
2. To have *working* sql parser, which I can then use for reformatter/beautifier/tidy type of program
3. To be able to write a tool that would point to common problems with queries (like: column without table alias in a query with joins, or join with conditions in where clause).
I’ve completed my own sql parser and formatter, please check it would be nice in everyday use. http://sites.google.com/site/sqlformatter/
Quick note to others:
the link Mak posted links to zip file, which contains EXE. No idea what is inside, I don’t run randomly downloaded executables from the internet, don’t have windows, and generally don’t trust code without sources available.
I share ur concept of getting anything from the internet. I posted that package uder this subject because I had same needs as yours when I decided to write this prog and I found its usefull on daily basis. Its windows executable, maybe others would be interested. Thanks for not deleting this advertisement 🙂