April 8th, 2009 by depesz | Tags: , , , , | 3 comments »
Did it help? If yes - maybe you can help me? Donate BTC to 19zPa5diT2LZqGtTi8f8bfApLn8rw9zBHx

Some years ago I learned of existence of (supposedly cool) POE framework for Perl. I tried to use it for some projects, but the learning curve proved to be fatal for my interest.

All the time I felt that POE is great, it's just that I'm too stupid to be able to actively use it.

Time passed by. Something like half a year ago, friend asked me if it would be possible for me to write a cool program – HTTP proxy for their MUD.

In case you're not familiar – MUD is a quite old type of game, that involves many users playing simultaneously on server (think: MMORPG) but without graphics.

All information is provided as text – location descriptions, actions, everything.

The thing is that MUD works as TCP server, accessible usually with telnet program, or some specialized programs which add capabilities like scripting. But the core game is played over open TCP connection between client and server. If the connection is lost, you can't play.

Now, this is in direct contrary to HTTP, which is stateless protocol – send request, get response, close connection. Of course there are things like Keep-Alive connections, or push channels, but they all seem to be “not cool".

So, we envisioned a proxy, that would act as http server, but pass information to and from MUD server, keeping connection between MUD and proxy open all the time.

Typical flow of data in it would be:

  1. proxy gets initial request from client (http)
  2. proxy opens connection to server, and returns session id to client
  3. whenever server “says" something, proxy buffers the data for client
  4. whenever client sends request – proxy returns all data from buffer to him
  5. if client provides “command" to be sent to mud, it is passed to mud using mud connection associated with given http-session-id.
  6. if server will close connection – proxy should keep this information for client, together with last (not yet received) messages from server, to be passed to client

If you ever heard about POE or anything like it (for example Twisted in Pythonland), you should see that description of the proxy matches perfectly what POE can (or at least should) do.

Given this, and my previous attempts at using POE, I decided to write it in POE – killing 2 birds with 1 stone – writing program for friend, and learning something new.

It took me longer than expected, but finally – it's done. First version is available to be downloaded and scrutinized.

Right now, it's not fully completed – i.e. there is no configuration file, logs, and so on, but the program itself works, and works quite nicely (at least – as far as my tests go).

In mud-proxy.pl (which is the proxy itself, test-server.pl is just some test tcp server I wrote to be able to test proxy), important parts are:

my $server = POE::Component::Server::HTTPServer->new(
'port' => 8181,
'handlers' => [
'/mud' => \&handle_requests,
'/' => new_handler( 'NotFoundHandler' ),
],
);

Which starts the HTTP part of the proxy on tcp port 8181, and adds handler for /mud url.

my $mud = POE::Component::Client::TCP->new(
'RemoteAddress' => '127.0.0.1',
'RemotePort' => '9000',
'Alias' => $client_id,
...

Which starts connection to MUD server, assuming it works on 127.0.0.1:9000.

Once running, you can request http://127.0.0.1:8181/mud to get data from server, and pass your input as cmd http parameters.

For example, sending request:

http://127.0.0.1:8181/mud?cmd=ls%20-l

will send “ls -l" command to mud server.

As far as commands go – proxy adds “\r\n" at the end of cmd, but only if it is not already there.

Of course this proxy can be used with any server, not only MUD. As long as the server uses TCP communication over defined port – it can be proxied to using mud-proxy.

As you will probably notice there are not much of security checks in the code – this is simply because the program (as it is now) is meant to be used as a backend, and be called from some front layer (php? mod_perl? django? whatever), and not directly web browser, though it is of course possible to add security measures, extensive logging, and basically anything that you could find useful.

Finally, I would like to express my sincere “THANK YOU" to all helpful souls on #poe channel on irc.perl.org that helped me whenever I felt lost (approximately every 15 seconds). Without you I wouldn't be able to write it.

  1. 3 comments

  2. # James
    Apr 10, 2009

    Coro/AnyEvent are way superior in terms of usability and performance. They just lack the numbers of dependent modules POE has, though with Coro::Select you can use many modules that use blocking selects, which you can’t with POE.

  3. # Anonymous
    Jun 8, 2010

    I have done similar things with POE. It works quite great, but POE does have problems scaling. When you go over 1000 sessions, garbage-collection of sessions often becomes a longwinded task that makes the entire program lag — so if you want to use this with many clients, I would recommend running more than one instance, such that each instance takes care of maybe 500 users at most. I did try these kind of things with up to 5000 connections to one POE process, but the process often started to hang for up to two minutes when it had to GC lots of sessions.

  4. Jul 6, 2010

    @james

    POE performance is often a factor of how it’s used. Programs using POE can perform very well when written with performance in mind. See http://gyazo.com/ecbcad673a0a47bb08c9b6b8c03b857b.png for an example.

    @anonymous

    Nine months ago I overhauled POE’s I/O dispatch and Session GC. I/O dispatch is about 130% faster. I can’t quote a number for session GC improvements because it’s difficult to measure in isolation. See my post at http://use.perl.org/~rcaputo/journal/39749

    Nevertheless, POE’s not too difficult to use with a limited number of sessions. In some cases, a singleton session can drive an entire program. If you can do this, it would totally eliminate session GC as your bottleneck.

    My latest project, Reflex, uses a single session to drive any number of objects. It also minimizes use of POE message passing, which is another source of overhead. Because of this, Reflex can run faster than POE even though it uses POE. As I said to James above, POE’s performance is a factor of how it’s used, and one of Reflex’s goals is to use POE well. See http://search.cpan.org/dist/Reflex/

Leave a comment