Installing Weblocks

Ok, now the important part, getting weblocks, patching it if needed, and getting its dependencies.

Getting weblocks:

cd ~/
# if you have weblocks you can update it with darcs pull
darcs get http://common-lisp.net/project/cl-weblocks/darcs/cl-weblocks


mkdir -p ~/.sbcl/systems
cd ~/.sbcl/systems
ln -s ~/cl-weblocks/*.asd .

The above ln command creates links to asd files so that weblocks can be easily loaded through slime.

Now, there are two dependencies that are hard to install the normal asdf way, so we'll install them manually.

mkdir -p ~/.sbcl/site/
cd ~/.sbcl/site/
# get 2 metabang packages
wget http://common-lisp.net/project/dynamic-classes/dynamic-classes.tar.gz
tar xvzf dynamic-classes.tar.gz
wget http://common-lisp.net/project/lift/lift_latest.tar.gz
tar xvzf lift_latest.tar.gz

Normally we should create links to the asd files inside ~/.sbcl/systems as we did for weblocks above. But it appears (confirmed) that when we do the following asdf installs, those links get created by some of the other metabang packages.

Also, while installing Gary King's metabang packages we do it in the following order: lift, metatilities-base, dynamic-classes -- we'll see this in slime, below.

At the time of this writing it was necessary to patch weblocks too. So first the patch, thanks to Bob Hutchison. Note that wget might not work with the patchname, so if the following doesn't work, use a web-browser to download the patch file and then copy it to the ~/cl-weblocks directory

cd ~/cl-weblocks
wget http://groups.google.com/group/weblocks/attach/0b57cd95a153dd1c/accommodate-metatilities-chages.patch?part=2
darcs apply accommodate-metatilities-chages.patch

Once darcs is finished your weblocks is up to date. Note: this tutorial refers to events from early June 2007 -- it's possible you may not need to do this patch. If you don't have darcs, get it with apt-get install darcs.

RUNNING EMACS + SLIME

Now, our first test run of lisp. These steps will become familiar to you as the incantations to start our environment. Optionally, first start screen (see subsection on screen in earlier post).

emacs -nw
M-x slime

You'll see slime load with sbcl and possibly some swank messages flash by, followed by the animation. If you see CL-USER you're in business, running the most powerful environment known to programmers (well, sort of!).

First we'll get some things weblocks needs, the dependencies are:
'( :closer-mop :metatilities :hunchentoot :cl-who :cl-ppcre :cl-json :puri :rt :tinaa :md5 :cl-fad :fare-matcher :cl-cont )

You could create the following file outside emacs too, but for argument's sake, let's do it here. C-x f is called 'visiting' a file and it creates one that doesn't exist.

C-x f ~/get-weblocks-deps.lisp

Emacs tips

While typing a command, C-g will usually get you out of it. C-h b shows you the current keybindings, C-h t runs the tutorial (too long), C-h c lets you describe a key: eg, try C-h c C-x C-c it should tell you the command that key runs, C-h k is the same but gives you more detail (try C-h k C-g it will tell you the key to run to remove the window). Occasionally, unwanted info windows will pop up (eg quite often after a compile). Some of them, like the compiler ones you can get rid of just by pressing q.

For other windows the following rule applies: If the cursor is in the unwanted window, type C-x 0, if it's in the window you want to keep, type C-x 1. Remembering this rule will save you a lot of frustration.

Now type the following:

(require 'asdf-install)
(asdf-install:install :closer-mop)

;Now get some packages from Gary King's metatilities.
;used to be '(:lift :metatilities-base :dynamic-classes))
(dolist (i '(:metatilities))
(format t "Metabang package: doing ~A" i)
(asdf-install:install i))

(defparameter *weblocks-deps*
'(:cffi :hunchentoot :cl-who :cl-ppcre :cl-json :puri :rt :tinaa :md5 :cl-fad :fare-matcher :cl-cont ))
(dolist (i *weblocks-deps*)
(format t "Doing ~A" i)
(asdf-install:install i))

Save the file

C-x s

Now we run the file using slime

C-c M-k

You should see some activity on the screen. An 'Install where?' dialog will be presented to you. Choose to install the files in your local system (option 2: Personal Installation). Also, when the loop runs, it will tell you which package it's installing ("Doing DYNAMIC-TEST-CLASSES Install where?"). If you run into trouble you can force an error by typing 0.

If there's an error, for example if you enter 0 instead of 1 or 2, you will be dropped into the debugger. A new window should open with the title *sldb sbcl/1* -- it will contain a list called restarts. To start from scratch, in general you can always choose the ABORT or TERMINATE-THREAD restart by typing the number next to it. You can use this as a way to break out of sticky situations.

In a similar vein, if you get stuck you can reset by typing C-x C-c which will close emacs. A warning message may be presented at the bottom of your emacs window that says active processes exist: type "yes" to quit anyway.

After you make your choice of where to install, a second dialog is presented to you related to GPG keys. Well, actually it's not a dialog, but rather that you get dropped into the debugger again because the GPG check will likely fail. Choose 0 to skip the GPG check and you will be fine. In general this is not a good practice, and after much digging around one could add the requisite GPG keys.

But it's a big hassle and we'll avoid it for now. To do things properly though, you should independently acquire the public keys of the lisp developers whose packages you want, and install them as trusted keys in your system. (TODO include pointers to good gpg tutorials here, preferably lisp specific: there is one on cliki or cl.net).

Now the package downloading and installation begins (you are connected to the internet, right?). It'll take some time and you'll be prompted for each package in your list, as well as all the packages those in the list depend upon.

Alert!

This is an excellent time to back up your install. Sometimes websites are down and packages are hard to get (this happens quite often with CL packages!). We will first delete any object files and then tar the rest.

cd ~/.sbcl
find . -name '*.fasl' | xargs rm
tar cvf ~/dotsbcldir.tar *
cd ~/cl-weblocks
find . -name '*.fasl' | xargs rm
tar cvf ~/clweblocks.tar *
cd ~/slime
find . -name '*.fasl' | xargs rm
tar cvf ~/slimeydir.tar *
cd ~/
tar cvjf all-lisp-webdev.tar.bz2 dotsbcl* clwebl* slimey*
# Copy this last file somewhere!

Please note problems in the comments so I can update these instructions with possible gotchas. Please don't post them to the list, since the procedure's been tested and it's not weblocks' fault if it doesn't work for some reason.

Running Weblocks for the first time

Let's assume you are no longer running emacs/slime (C-x C-c if you are).


0. Loading incantations

emacs -nw
M-x slime

at CL-USER> type

, load-system weblocks

(you can use tab completion and up-arrow after the comma)

Weblocks will load, because the weblocks.asd file is in a standard place. If this is the first time, some .fasl files will get created and it'll take some time. Subsequent loading will be faster since the compilation will have been done the first time around.

A compile messages window will pop up -- try using C-x 0 or q to get rid of it.

Now again at CL-USER> type

"(weblocks:start-weblocks " remember to type the space!

Now we see slime telling us what the function signature is. In the mini-bar window (TODO called what?) you'll see
(start-weblocks &key debug (port 8080) &allow-other-keys &rest keys)
this means that the function takes two key arguments, debug and port, and that port is by default 8080 unless specified.
&allow-other-keys means that if you override this function (TODO check) it will accept other keys even though this particular version in weblocks doesn't use them for anything.
&rest args (or keys, in this case) is a common idiom to group unspecified other arguments into a simple list that you can iterate over, or pass to another function.
finish up with (weblocks:start-weblocks :debug t :port 8080)

where :debug indicates it is a keyword symbol.

A hunchentoot server object is returned, this server is now talking on port 3000.
Open a browser and go to localhost:3000 (this url is convenient for us because that's where our failed RoR experiments lie!)
You should see the weblocks welcome page.

1. Run the standard weblocks demo (note other demos and dependencies)

Now we'll run the standard weblocks demo. There is also a clsql and elephant demo, but they have other dependencies like postgres etc.

Since the asd file for the demo is not in the standard directory, we have to tell ASDF where to find it. We do this by pushing its filename (aka pathname, the full canonical file path) onto a list maintained by ASDF. #p"/a/b/c" returns the pathname for the file c in the directory /a/b/ -- at least on unices.

type the following at your CL-USER> prompt:

(push #p"/home/lisper/cl-weblocks/examples/weblocks-demo/" asdf:*central-registry*)

lisp replies with the value of asdf:*central-registry* (which is a variable *central-registry* inside the namespace asdf)
The * doesn't signify anything special, other than that it's a global variable (lisp variables and functions can pretty much have any character as part of the name). A similar convention is used for variables called +i-is-a-variable+ too.. TODO fact-check.

You can see the value again. Just type asdf:*central-registry* at the repl.

You can see previous values at the repl by either using * by itself on a line (a repl shortcut unrelated to the above discussion) or by pressing M-p M-n to move back and forward in the command line history. you can also search for all previously typed expressions beginning with, for example, "(mooshu" by typing "(mooshu" (without quotes) followed by M-p [that is, type the expression to search, then press the meta and p keys together.

Anyway now you can see that the global variable is set to a list of places to check, and the one we added is first.

Now we can do

,load-system weblocks-demo
(use tab to see other ASD systems that ASDF found in the same directory)

to load the demo. Now we run the server with the earlier start-weblocks or with the equivalent

(weblocks-demo:start-weblocks-demo :debug t :port 8080)

which is a function in the weblocks-demo namespace that probably just calls the original function.
heck, we can check this. type (weblocks-demo:start-we[now press TAB to complete] [now press M-. (alt and period)]
voila, we jumped to the function definition! We can see it just calls the start-weblocks function from earlier, with the args it was given (which it doesn't even check, it leaves that to the original function to figure out!) Now press M-, to jump back (twice to go back to the repl)

Play around with M-. -- for example if you go back to the start-weblocks definition, at the bottom go to hunchentoot:session-value and press M-. a miniwindow opens up with some options. this is a good place to use C-x 0 :-)

Note: if you have already run weblocks:start-weblocks then weblocks-demo:start-weblocks-demo will return NIL, indicating it didn't do anything. stop the original hunchentoot server with weblocks:stop-weblocks first; and now the server runs just fine, and its value is returned, as before.

2. Make your own toy app

Weblocks now comes with a few functions to easily create the webapp file and directory structure.
Let's go back to the repl with

f12 r

Now type

,load-system weblocks-scripts
(weblocks-scripts:make-application 'mywebapp "/home/lisper/")

this will create a new webapp and its directory structure in /home/lisper.. you can leave off the second argument if you started your emacs in /home/lisper

however, the following is an error and will drop you into the debugger.

(weblocks-scripts:make-application "mywebapp" "/home/lisper/")

as the minibuffer function signature indicates, the second argument is optional .. if not included by default some notion of current directory will be used (you can find out what by doing M-. and seeing what the function does!). A great explanation of functions (and especially &key and &optional) is given at http://gigamonkeys.com/book/functions.html (chapter t) Peter Seibel's book is totally worth reading -- and very highly recommended by all lispers. It presents a modern way to look at lisp and has an eminently understandable way of explaining things, not to mention a great conversational tone, excellent fully fledged examples, copious explanatory footnotes, full implementation of a bayesian filter, mp3 shoutcast server and mp3 id-tags parsing library, complete unit test suite -- all in portable common lisp. His website also has lispbox, a lisp in a box probably most useful to marooned win32 users. There don't seem to be many (any?) weblocks users on win32 though.

Making (+ sbcl slime weblocks) load like the blazes.

This is one of the cooler tricks. One is supposed to keep the emacs session around long enough that speed of loading isn't an issue. However, this little hack can be quite useful in practice, say if you're on a laptop or open and close your emacs often.

What it does is load swank and weblocks once into a running lisp image, and then save that image. Subsequently when slime runs sbcl it loads the image. All of weblocks loads in a flash, at the expense of about 80MB of disk space (on a 64-bit linux box).

Quit any running emacs/slime. Go to the ~/.sbcl directory with

cd ~/.sbcl

Create this file:

cat > mkcore1.lisp
(require :asdf)
(push #p"/home/lisper/slime/" onto asdf:*central-registry*)
(asdf:operate 'asdf:load-op :swank)
(asdf:operate 'asdf:load-op :weblocks)
(load "/usr/share/common-lisp/source/slime/swank-loader.lisp" :verbose t)
;; a better way to do this is below
; (progn (load "/usr/share/common-lisp/source/slime/swank-loader.lisp" :verbose t)
; (funcall (read-from-string "swank:start-server") "/tmp/slime.7144" :coding-system "iso-latin-1-unix"))
;(sb-ext:save-lisp-and-die "sbcl.core")
(sb-ext:save-lisp-and-die "sbcl.core" :toplevel (lambda () (swank:create-server :port 4005) (sb-impl::toplevel-init)))
(quit)
^D

Before we proceed further though, we have to ensure that swank.asd exists in the default load path. As an exercise, this is left to the reader. There are two ways to do it!
[go to ~/.sbcl/systems and ln -s /home/lisper/slime/swank.asd . or push #p"/home/lisper/slime/" onto asdf:*central-registry* in the code above]

Now at a terminal go to ~/.sbcl and type

sbcl --load mkcore1.lisp

and watch the messages flash by. If all goes well you will be left with a sbcl.core file in the directory. Type ls -ltrh to check that it's there.

Now we open our ~/elisp/lisper.el file to enable loading the core. Find the line below "replace core" and uncomment it. Comment out the "original core" line with a leading semicolon. You're done! (Note you can rename cores and create different cores with stuff preloaded)

Now restart emacs and slime as before. An uber-functional web development environment in under three seconds! How cool is that?

Summary

This slightly long introduction's taken you from a fresh mac os x or linux user's system to an installed web development environment. Post questions here (there are subsequent posts for beginner and other questions).

Future plans include explaining the weblocks framework and keeping up to date with changes. The next few posts should be about with-flow, login and widget trees because as of now these seem most important to getting up and running the weblocks way. Meanwhile explore the demos, and the tests directory of the weblocks source.