roxbase: Lisp Web Templating

roxbase is a minimal and very simple Template System for Lisp. It isn't yet feature-complete and first and foremost intended for my own use. I have only 'tested' (used) it under .^[1]

Purpose

What do I mean by Template System? I wanted something similar to Edi Weitz' (of fame) , but more akin to ASP.NET than that Perl templating module which inspired aforementioned HTML-TEMPLATE.

So the basic idea is to embed Lisp code in text files (think HTML pages) and process those files so that each Lisp expression be replaced with the result of evaluating it.

How it works

The most obvious use for roxbase is within a web context. The system integrates well with Hunchentoot, this website for one runs on Hunchentoot and roxbase. However, you can use within any other kind of context. roxbase only provides some basic string processing logic as described below.

Once you have set it up, you can feed roxbase templates such as this:

<a href="<%= (my-url-function (random 50)) %>">
<%@ 'link-caption %>
</a>

Types of template expressions

There are two kinds of expressions that are being processed by roxbase:

Name	Description	Example
Inline Expressions	Contains self-contained Lisp expressions.	Two and two makes <%= (+ 2 2) %>
Mix-In Expressions	Mixes Lisp code and template contents arbitrarily.	Let's loop <b>ten</b> times: <? (dotimes (val 10) ?> <div>   times <i>two</i>: <?= (* val 2) ?> </div> <? ) ?> That's it, thanks!

The process-template function first processes all mix-in expressions it finds, then all inline expressions. You can have your mix-in or inline expressions generate new mix-in or inline expressions on the fly, making for neat and possibly hard-to-debug, macro-like, recursive templating: if as the result of processing, new mix-in or inline expressions are present in the intermediate output, it repeats this process until the template has been completely processed (tail-recursively, of course).

(Beware of creating endless loops when dynamically generating new mix-in or inline expressions!)

Inline Expressions (InLexes)

The process-template function processes all inline expressions (InLexes) it finds in the template it is given. InLexes start with <% (unless you change the value of +inlex-start-tag+) and end with %> (unless you change the value of +inlex-end-tag+). In addition, <% must be followed directly by a character that identifies the kind of InLex you're writing. The following InLexes are already built into roxbase. Needless to say, you can define your own. Only the first three InLex handlers really come with their own specific evaluation logic, the others translate into more complex InLex expressions of <%= or <%%.

	Name:	Description:	Example:
<%=	Eval	Replaced with the result of evaluating the enclosed Lisp expression.	<%= (+ (random 20) (random 40)) %>
<%%	Def	No output is generated. Can be used for defuns and other definitions. Your definitions are available to Eval InLexes. In contrast to Eval InLexes, the args hashtable is not available. Pass it as an argument to your functions if you need to access the template arguments.	<%% (defun rootpath (args)   (gethash 'rox-rootpath args)) %> Root path of this web app: <%= (rootpath) %>
<%!	Comment	No output is generated. Content is ignored and discarded.	<%! (needs-fix) %>
<%~	Load	No output is generated. Load .lisp / .fasl code files. Unless an absolute path (starting with '/') is specified, the search path is considered relative to your root-path.	<%~ "code/my-page-tools"     "code/my-utils" %>
<%?	Query	If you're using Hunchentoot, replaced with the value of the URL query string parameter with the specified name.	<%? "foo" %>
<%&	Form	If you're using Hunchentoot, replaced with the value of the POSTed form parameter with the specified name.	<%& "foo" %>
<%$	Session	If you're using Hunchentoot, replaced with the value of the session variable with the specified name.	<%$ "foo" %>
<%|	Header	If you're using Hunchentoot, replaced with the value of the request header with the specified name.	<%| "Accept-Content-Type" %>
<%@	Argument	Replaced with the value associated with the specified key in the template's args hash table.	<%@ "my-arg" %> or <%@ 'my-other-arg %> or whatever you chose to use for a hash key in the args table.
<%"	HTML Encoder	Replaced with the result of passing the enclosed Lisp expression to Hunchentoot's escape-for-html function.	<pre><%" "Sample code:     <hr />" %></pre>
<%'	JavaScript Encoder	Replaced with the result of passing the enclosed Lisp expression to the escape-for-js function.	alert("<%' "'Bobo'     said \"Hi\"." %>);
<%:	Safe Encoder	Replaced with the result of passing the enclosed Lisp expression to the escape-for-safe function, which accepts multiple strings, concatenates them with *rox-escape-char* (by default, the underscore) and replaces all non-alphanumeric characters in the resulting string with said escape character.	<a name="<%: "why not?" %>"> Anchor</a>
<%#	Sub Template	Replaced with the result of processing the specified template with the specified arguments (those will be available to the template in its args hash table).	<%# "/controls/footer.html" ("copyright" "2008") ("color" "blue") %>

Getting started

Before you can use it, roxbase needs to be initialized. Call the following function:

server-startup

• name — This could be the name of your package or any other name. If you want server-startup to start a Hunchentoot instance, this will be the name of your server.
• port — If you do not want server-startup to start a Hunchentoot instance, use NIL; otherwise, specify the port of your Hunchentoot instance.
• pathname-root — the absolute path of the directory serving as the base path for other relative path specifications used within templates or during a template processing request.
• script-path-handler: designates a function that accepts three arguments and returns the path (absolute or relative to pathname-root) to the file that should be processed by roxbase.
  • script-name — the request path for the processing request
  • args — the Arguments hash table for the processing request (template processing arguments are described further below)
  • subtemplate-p — T if this processing request was invoked recursively during a parent template processing.
• &key foldername-static — the path (relative to pathname-root) for files that should not be template-processed by roxbase.
• &key debug — unless port is NIL, sets the Hunchentoot variables *show-lisp-errors-p* and *show-lisp-backtraces-p* to the specified value (should be Boolean — NIL or T).
• &key scriptpackage — unless NIL, all <%% directives will be prepared with an (in-package) call before processing, where the package is either the value of scriptpackage or of name, if scriptpackage's value is T.

As described above, the port argument implicitly controls whether server-startup starts up and returns a new Hunchentoot server instance. If this is the case, roxbase sets up its own Hunchentoot request handlers that either simply return a file unprocessed if the request path fits the directory specification (relative to pathname-root) in foldername-static, otherwise, it invokes the specified script-path-handler function and processes the file represented by the path (relative to pathname-root) returned by the function call.

server-shutdown

After you're done with your template processing, you can call the server-shutdown function to clean things up. If your previous server-startup call initialized a new Hunchentoot server instance (that is still stored in the global variable *rox-server*), the service is stopped. Afterwards, the following global hash tables are cleared:

• *rox-inlex-cache* — contains cached compiled equivalents of successfully parsed inline expressions (InLexes)
• *rox-inlex-handlers* — contains all known InLex handlers (InLexes and InLex handlers are described further below); pre-populated with the roxbase system InLex handlers by server-startup
• *rox-settings* — pre-populated by server-startup with the following default values:
  • 'fn-html-condition-handler (a function designator, by default #'default-html-condition-handler, that produces output to indicate an error in place of a processed InLex and accepts three arguments:
    • condtype — either "EVAL", "COMPILE" or "RUNTIME", depending on when the InLex error was encountered
    • condsrc — the complete InLex that caused the error
    • condmsg — the error message)
  • and 'eval-compile-cache-handlers-p (the string of InLex handler identifiers whose expressions should be compiled and cached in *rox-inlex-cache*).

Invoking template processing requests

handle-request, create-args-table, process-template, load-template

If you're using roxbase with Hunchentoot, each non-static request is handled by handle-request, which returns a string that is the result of processing the file indicated by the script-path-handler you specified when you called server-startup. You can call handle-request yourself, it has the same signature as a script-path-handler. In case you ever need to do just that, pass the following arguments:

• script-name — pass any value that your script-path-handler will transform into the path (absolute or relative to pathname-root) of the file to be processed
• args — pass a hash table freshly created by the create-args-table function:
  • pathname-root — just pass the same path name you passed server-startup
  • script-path-handler — just pass the same function you passed server-startup.
  • parent-args — just pass NIL
  • root-args — just pass NIL.
• subtemplate-p — just pass NIL (this should only ever be T if called automatically from a parent template processing request)

If handle-request finds the specified file, it loads all its raw, unprocessed string contents into memory and passes it to the process-template and returns the result of that call; otherwise, it returns NIL and, when using Hunchentoot, sends a 404 status code prior to returning.

Unless called manually, handle-request is called either in response to a Hunchentoot web request or by the load-template macro, which is a syntactically compact way to recursively load sub-templates during a parent template processing request.