STARTER-PACK - My personal Common Lisp "newbie" guide


 

Abstract

This is something I started in 2006 and at that time it seemed to make sense. Now (2011), you should try Zach Beane's Quicklisp instead. I'm not planning to maintain the starter pack anymore.

This is an attempt to help Common Lisp newcomers getting started. The aim is to quickly set up a comfortable working environment which includes a couple of useful and/or popular open source libraries. It is targeted at the Microsoft Windows platform (but see below) and obviously heavily influenced by the author's personal preferences.

If you're using this document and the accompanying application, I'd like to hear from you. Whether you like it or not, feedback is always appreciated. See my homepage for my email address.

Screen shot

[The current version is 0.6.2. Ignore the screen shot if it shows another number.]

 

Contents

  1. Introduction
  2. Prerequisites
  3. Installation
  4. How to use your new environment
  5. Usage examples
  6. What is this ASDF thingy?
  7. Your own code
  8. What does the starter pack do?
  9. Potential problems
    1. Wrong URLs
    2. Wrong dependencies
    3. One of the libraries doesn't work
    4. Proxies
  10. Customization
  11. FAQ
    1. Why only Windows?
    2. Why only LispWorks?
    3. Why not SLIME?
    4. Why not ASDF-INSTALL?
    5. Why not Lispbox or Lisp in a Box?
    6. Why is library "foo" not included?
  12. Copyright
  13. License

 

Introduction

It has happened to me a couple of times that people who wanted to learn Common Lisp asked me for help or advice because they struggled with the basics, or rather with seemingly peripheral problems that prevented them from doing the "real stuff." These problems are usually related to setting up a working environment and to using open source libraries. I faced very similar issues when I started learning Lisp in 2000, and although the situation has improved since (you'll find much more introductory material on the web, and the library situation is significantly better), to many it still seems hard to reach a state where you can concentrate on learning the language and write a simple program.

I usually tried to help by setting up a little "starter pack" which (I think) was easy to use and included the libraries the person wanted to deploy in a state ready for immediate usage. My experience with this "starter pack" has been quite good, and so I decided to polish it a bit, make it customizable, and publish it, so I can point people to this website the next time I'm asked.

Feel free to use this document and the accompanying little application as you see fit, but be aware that all of this will only really work if you're willing to follow my lead, specifically if you're using the same Lisp implementation I do - see FAQ for more about this topic.
 

Prerequisites

I assume that you are interested in Common Lisp and want to learn it. This document is not intended to convince you that Lisp is a great programming language. If you need some encouragement first, start with Pascal Costanza's very good guide.

I also take for granted that you've found good introductory material which explains the syntax and semantics of Common Lisp. If you haven't done so already, you should definitely take a look at Peter Seibel's great tutorial Practical Common Lisp. I'd recommend buying the dead-tree version, but note that it's also available as a free download from Peter's website and (as a PDF) from the Apress website.

Finally, if for religious or political reasons you only want to use "free" software and Windows and/or LispWorks don't fit with your definition of free, then this text isn't for you.

As for technical requirements, you should use Windows 2000, XP, or Vista with all updates and service packs installed. What I describe here most likely won't work with older Windows versions - you have been warned...
 

Installation

Here's a step-by-step explanation of how to set up your environment: Note that if you use LispStarterPack.exe again, it will completely remove and replace the directory it has written, i.e. the application is not intended to be used more than once to add libraries to the ones you already have. If you're not sure which libraries you want to have, it doesn't hurt to press the Check all button if you have a fast Internet connection.
 

How to use your new environment

Now start LispWorks and, as your very first step, use the File menu to Load... (and not only Open...) the file start.lisp that got installed in a directory called "Lisp Libraries" within your "My Documents" folder. (Of course, depending on your language settings, "My Documents" might have another name, for example "Eigene Dateien" in German.) LispWorks will now start to work frantically to compile some libraries. Just wait until it's finished. (Don't worry - this lengthy compilation only happens the first time after you've installed the libraries.)

It is assumed that you don't use LispWorks' Windows editor emulation, by the way. If you do, many of the key bindings described below won't work.

Unfortunately, you will have to load start.lisp every time you start LispWorks. LispWorks knows the concept of an initialization file which is read on startup and you could use start.lisp as your initialization file - but one of the deliberate limitations of the Personal Edition is that it doesn't read initialization files. If this annoys you very much, you should consider to use the Windows Scripting Host to automate this task - see Bill Clementson's blog for hints.

It is generally a good idea to familiarize yourself with new tools you plan to use extensively. In this case, this means that you will have to learn how to use LispWorks. Spend a couple of days (yes, I said days!) playing with the editor and the various other tools the IDE has to offer. I'd suggest to read at least the following parts of the LispWorks documentation:

Note that the complete LispWorks documentation is on your local hard disk if you followed the instructions above.
 

Usage examples

Here are some examples, in no particular order, you should probably try out in order to get familiar with your environment:
 

What is this ASDF thingy?

A major goal of this "starter pack" is to initially shield you from the complexities of ASDF. But sooner or later you will have to learn what it is and how to use it. So, here we go:

ASDF is a system definition facility. It can be used, amongst other things, to define which source files belong to a library and in which order they have to be compiled and loaded. In this respect it is pretty similar to make. There are other, similar systems like MK:DEFSYSTEM or LispWorks' Common Defsystem, but the dominant system for open source libraries nowadays is ASDF.

An ASDF system is described by a file with the .asd suffix. If you instruct ASDF to load a system called "foo", it will try to find the foo.asd file and proceed from there. If you plan to only use ASDF to load libraries others have written, you only need to know how to tell ASDF where to look for these "system definitions." This is done via the special variable ASDF:*CENTRAL-REGISTRY* which is a list of pathname designators denoting directories. (I'm simplifying a bit. See the ASDF manual for all the gory details.)

On a Unix-y platform, most ASDF users really have one central registry which is a directory to which they symlink all the system definitions they have on their hard disk. Since Windows doesn't have symbolic links (yeah, yeah, I know about "reparse points" aka "junction points"), you have to use another approach. The approach the "starter pack" uses is to put all libraries into one directory, parse this directory for sub-directories which contain system definitions, and add these sub-directories to ASDF:*CENTRAL-REGISTRY*. This happens when start.lisp is executed.

So, in order to add new Lisp libraries you've downloaded manually, just unpack them into the "Lisp Libraries" folder mentioned above. ASDF will know about this library after you've restarted LispWorks and executed start.lisp again. As an alternative, you can evaluate

(cl-user::update-asdf-central-registry)
in the current session.

This works fine unless you have different versions of the same library in the "Lisp Libraries" folder. So, don't do that...

For a slightly different approach to using ASDF on Windows see the Windows note here.
 

Your own code

For the reasons explained above, it is recommendable that you also use ASDF for your own stuff. Here's a very simple recipe which you can use to organize your own code until you feel ready for more complex setups. In this example, we're creating a system called "foo": Now you can begin to work on your code. Always start by loading your system. This will ensure that all required libraries are loaded and, as a consequence, that things like symbol completion, display of argument lists, and indentation work correctly.

If you edit code, don't (re)define functions in the listener. Instead, edit directly within the source files and use the editor's compilation commands.

For a more detailed explanation of ASDF see Mario Mommer's howto.
 

What does the starter pack do?

Once you've understood how ASDF works, it is pretty easy to understand what LispStarterPack.exe does: It downloads the libraries you've specified and unpacks them into the "Lisp Libraries" folder. It also puts ASDF itself there as well as the file start.lisp which is pulled from my my website. start.lisp, which is supposed to be loaded after you've started LispWorks, tells ASDF where to look for system definitions and sets up some stuff for LW-ADD-ONS.

If you've selected CLSQL amongst the libraries you want to download, the DLL for SQLite will also be downloaded and start.lisp will modify the PATH environment variable so Windows will find this DLL while LispWorks is running. (If you want to play with CLSQL, you can also connect to an ODBC source or to a PostgreSQL database using the socket back-end. For other types of SQL databases, like Oracle or MySQL, you will have to install the corresponding DLLs yourself.)

The source code for LispStarterPack.exe including this website is available from http://weitz.de/files/starter-pack.tar.gz and the program can download and install its own source code in case you want to inspect it. (Load the starter-pack system and call the function STARTER-PACK:MAIN.)

Note: Currently (as of August 16, 2006) the program has to use an unreleased version of ZIP which I keep at http://weitz.de/files/zip-cvs-2006-08-16.tar.gz on my website. The latest release version doesn't work with Windows and/or LispWorks. This will be changed as soon as a new release is out. See also next section.
 

Potential problems

Due to the approach I've chosen, several things can potentially go wrong:

Wrong URLs

The URLs for the libraries are hard-coded into the application. So, if the author decides to change the URL, you won't be able to download this library anymore. If you see this happen, please contact me so I can release a corrected version of LispStarterPack.exe.

For a better, albeit not perfect approach to deal with this problem, see ASDF-INSTALL.

Wrong dependencies

The same applies to library dependencies which might change without me noticing. If something goes wrong, you should always check first if you're using the newest version of this starter pack, though.

One of the libraries doesn't work

Most of the URLs are chosen to automatically pick up the newest version of the corresponding library. Of course, it can happen that a newer version breaks or simply doesn't work with Windows and/or LispWorks anymore. You can email me if this happens to you, but I probably can't do anything about it.

Proxies

If you are using a proxy to connect to the Internet, then LispStarterPack.exe probably can't download anything. It will query the Windows registry for the static proxy settings and adhere to them, but it won't be able to cope with per-connection proxy settings or with proxies which require authentication.
 

Customization

If you have a file called .lw-init.lisp (note the initial dot) in your home directory, this will be automatically loaded after start-lisp, so you can put arbitrary initialization code her.

You can also add libraries to the ones that are hard-coded into the application if you want. Likewise, you can overwrite the default download URLs. This is done by creating a file called config.lisp in the same directory where the application resides. The application will read this file when it starts up and it will add library definitions found therein to the ones it already knows. A library definition in this case is a Lisp list with three or more elements where the second and third elements are strings and all other elements are keywords. The first element is the name of the library which will be shown in the graphical user interface next to the checkbox and which is used to uniquely identify the library. The second element is a textual description which will be used for the tooltip. The third element is the download URL. And the following elements (if any) denote other libraries this one depends on. These libraries must also be known to the application or it will refuse to start.

An example config file could look like this:

(:cl-wbxml "WBXML parser."
           "http://weitz.de/files/wbxml.tar.gz"
           :flexi-streams)

(:uffi "Portable access to C libraries."
       "http://files.b9.com/uffi/uffi-1.5.15.tar.gz")
The first expression adds the CL-WBXML library to the ones which should be downloaded, and the second one replaces the default download URL for the UFFI library with one that always loads version 1.5.15. The second expression replaces the original UFFI definition because the keyword :UFFI uniquely identifies this library.

Download URLs must always be HTTP or HTTPS (see this remark about SSL) URLs, and the types of the archives that are downloaded must be .zip, .tar, .tar.gz, or .tar. Libraries added this way must obviously have an ASDF system definition in order to be useful. Most code you will find at (for example) the Common Lisp Directory, CLiki, and Common-Lisp.net will have one.

For another example for a config file see the lispbuilder project.
 

FAQ

No, these aren't really frequently asked questions - nobody has asked any questions yet. But you probably want to ask some of these questions once you've read the website, so why not give an answer in advance... :)

If you don't know what I'm talking about below, just skip this section.

Why only Windows?

The original "starter pack" this document is based on was created for Windows users so I simply decided to leave it like that. There are actually two reasons for this decision: First of all, it involves much more work to prepare something that works equally well on a variety of operating systems. I don't have the time to do that. Secondly, I think that the situation for Lisp newbies who are on Linux or OS X is significantly better than for Windows users: Lisp open source hackers seem to favor these platforms and thus support for them is better, Emacs (and thus SLIME) works better than on Windows, and you can even expect "native" packaging systems like Debian's apt, Gentoo's emerge or OS X's Fink to help with Lisp installations.

For Linux, you should also check out clbuild. For OS X, read Bill Clementson's blog entry.

Why only LispWorks?

LispWorks is the Common Lisp implementation I use and that I'm familiar with. It is much easier for me to prepare something that works with LispWorks than to prepare something that works with every Lisp that can potentially run on Windows. I'm convinced that LispWorks provides a great environment to work with and also to learn the language, but of course that doesn't imply that I would discourage you from using something else like, say, Allegro CL, or the new Windows port of SBCL - it's just that this document ain't for you in this case.

Why not SLIME?

I really like SLIME. I think it's a great Common Lisp IDE, and I always use it if I have to work with Lisps other than LispWorks. However, I think that the LispWorks IDE is more accessible to newbies and that it provides a lot of niceties SLIME doesn't have (yet). Plus, if you don't use SLIME, you aren't forced to install Emacs which, as much as I like it, is kind of an impurity on Windows. Finally, there are some unresolved issues with SLIME and LispWorks on Windows.

Why not ASDF-INSTALL?

I originally started to port ASDF-INSTALL to Lisps other than SBCL and to write a little tutorial for it. However, ASDF-INSTALL is kind of biased towards a Unix-y platform, and while it is possible to make it work with Windows, my experience has been that even with the tutorial people had difficulties using it. Furthermore, I don't use ASDF-INSTALL myself. I think that once you're familiar enough with ASDF, you don't need it, and it's easier to just download libraries manually and put them where you want to have them. Likewise, I expect the readers of this document to get rid of the "starter pack" sooner or later.

Why not Lispbox or Lisp in a Box?

Lispbox and Lisp in a Box are cool, but they don't work with LispWorks. See also my remarks about SLIME above.

Why is library "foo" not included?

Your favorite open source library is not included in the list provided by LispStarterPack.exe? Why is that?

Well, I selected libraries according to the following criteria: They have a reasonable license, I know them and have worked with them, they work with Windows, they use ASDF, they don't use the foreign function interface (FFI), they don't duplicate functionality provided by other libraries in the list. (CLSQL was allowed to violate the point about not using the FFI because it's pretty popular, but otherwise I didn't fancy dealing with DLLs.)

If you think a specific library should definitely be added, contact me. Or use the customization option.
 

Copyright

Copyright © 2006-2011 Dr. Edmund Weitz. All rights reserved.
 

License

Redistribution and use of this tutorial in its orginal form (HTML) or in 'derived' forms (PDF, Postscript, RTF, and so forth) with or without modification, are permitted provided that the following condition is met:
Redistributions must reproduce the above copyright notice, this condition, and the following disclaimer in the document itself and/or other materials provided with the distribution.
IMPORTANT: This document is provided by the author "as is" and any expressed or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the author be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this documentation, even if advised of the possibility of such damage.

$Header: /usr/local/cvsrep/starter-pack/doc/index.html,v 1.88 2011-02-11 19:56:07 edi Exp $

BACK TO MY HOMEPAGE