CL-FAD - A portable pathname library for Common Lisp


 

Abstract

CL-FAD (for "Files and Directories") is a thin layer atop Common Lisp's standard pathname functions. It is intended to provide some unification between current CL implementations on Windows, OS X, Linux, and Unix. Most of the code was written by Peter Seibel for his book Practical Common Lisp.

CL-FAD comes with a BSD-style license so you can basically do with it whatever you want.

Download shortcut: http://weitz.de/files/cl-fad.tar.gz.


 

Contents

  1. Download and installation
  2. Supported Lisp implementations
  3. The CL-FAD dictionary
    1. Querying files, directories and pathnames
      1. directory-exists-p [function]
      2. directory-pathname-p [function]
      3. file-exists-p [function]
      4. pathname-absolute-p [function]
      5. pathname-equal [function]
      6. pathname-relative-p [function]
      7. pathname-root-p [function]
    2. Manipulating pathnames
      1. canonical-pathname [function]
      2. merge-pathnames-as-directory [function]
      3. merge-pathnames-as-file [function]
      4. pathname-as-directory [function]
      5. pathname-as-file [function]
      6. pathname-directory-pathname [function]
      7. pathname-parent-directory [function]
    3. Traversing directories
      1. list-directory [function]
      2. walk-directory [function]
    4. Temporary Files
      1. open-temporary [function]
      2. with-output-to-temporary-file [macro]
      3. with-open-temporary-file [macro]
      4. *default-template* [variable]
      5. cannot-create-temporary-file [condition]
      6. invalid-temporary-pathname-template [condition]
      7. missing-temp-environment-variable [condition]
      8. temporary-files [logical pathname host]
    5. Modifying the file system
      1. copy-file [function]
      2. copy-stream [function]
      3. delete-directory-and-files [function]
    6. path [package]
  4. Acknowledgements

 

Download and installation

CL-FAD together with this documentation can be downloaded from http://weitz.de/files/cl-fad.tar.gz. The current version is 0.7.2.

CL-FAD comes with simple system definitions for MK:DEFSYSTEM and asdf so you can either adapt it to your needs or just unpack the archive and from within the CL-FAD directory start your Lisp image and evaluate the form (mk:compile-system "cl-fad") - or (asdf:oos 'asdf:load-op :cl-fad) for asdf - which should compile and load the whole system. Installation via asdf-install should as well be possible. Plus, there are ports for Gentoo Linux thanks to Matthew Kennedy and for Debian Linux thanks to René van Bevern.

If for some reason you can't or don't want to use MK:DEFSYSTEM or asdf you can just LOAD the file load.lisp.

The latest version of the source code lives in the github repository edicl/cl-fad If you want to send patches, please read this first.
 

Supported Lisp implementations

The following Common Lisp implementations are currently supported:

I'll gladly accepts patches to make CL-FAD work on other platforms.
 

The CL-FAD dictionary

Querying files, directories and pathnames


[Function]
directory-exists-p pathspec => generalized-boolean


Checks whether the file named by the pathname designator pathspec exists and if it is a directory. Returns its truename if this is the case, NIL otherwise. The truename is returned in directory form as if by PATHNAME-AS-DIRECTORY.


[Function]
directory-pathname-p pathspec => generalized-boolean


Returns NIL if pathspec (a pathname designator) does not designate a directory, pathspec otherwise. It is irrelevant whether the file or directory designated by pathspec does actually exist.


[Function]
file-exists-p pathspec => generalized-boolean


Checks whether the file named by the pathname designator pathspec exists and returns its truename if this is the case, NIL otherwise. The truename is returned in "canonical" form, i.e. the truename of a directory is returned in directory form as if by PATHNAME-AS-DIRECTORY.


[Function]
pathname-absolute-p a => result

Returns true if a is an absolute pathname. This simply tests if a's directory list starts with :ABSOLUTE


[Function]
pathname-equal a b => result

Returns true if a and b represent the same pathname. This function does not access the filesystem, it only looks at the components of the two pathnames to test if they are the same (though by passing both a and b to probe-file one can make this function test for file 'sameness'.

Equality is defined as:

If any of these tree conditions is false for any of the components in a and b then a and b are different, otherwise they are the same.

NB: This function does not convert name strings to pathnames. So "foo.txt" and #P"foo.txt" are different pathnames.


[Function]
pathname-relative-p a => result

Returns true if a is a relative pathname. This simply tests if a's directory starts with :RELATIVE.


[Function]
pathname-root-p a => result

Returns true if pathname is the root directory (in other words, a directory which is its own parent).

Manipulating pathnames


[Function]
canonical-pathname pathname => result

Remove reduntant information from PATHNAME.

This simply walks down PATHNAME's pathname-directory and drops "." directories, removes :back and its preceding element.

NB: This function does not access the filesystem, it only looks at the values in the pathname and works on their known (or assumed) meanings.

NB: Since this function does not access the filesystem it will only remove :BACK elements from the path (not :UP elements). Since some lisps, ccl/sbcl/clisp convert ".." in pathnames to :UP, and not :BACK, the actual utility of the function is limited.


[Function]
merge-pathnames-as-directory &rest pathnames => result

Given a list of, probably relative, pathnames returns a single directory pathname containing the logical concatenation of them all.

The returned value is the current directory if one were to cd into each of pathnames in order. For this reason an absolute pathname will, effectively, cancel the affect of any previous relative pathnames.

The returned value's defaults are taken from the first element of pathnames (host, version and device).

NB: Since this function only looks at directory names the name and type of the elements of pathnames are ignored. Make sure to properly use either trailing #\/s, or pathname-as-directory, to get the expected results.

Examples:

  (merge-pathnames-as-directory #P"foo/" #P"bar/") == #P"foo/bar/"

  (merge-pathnames-as-directory #P"foo/" #P"./bar/") == #P"foo/./bar/"

  (merge-pathnames-as-directory #P"foo/" #P"/bar/") == #P"/bar/"

  (merge-pathnames-as-directory #P"foo/" #P"/bar/" #P'quux/file.txt) == #P"/bar/quux/"


[Function]
merge-pathnames-as-file &rest pathnames => result

Given a list of, probably relative, pathnames returns a single filename pathname containing the logical concatenation of them all.

The returned value's defaults are taken from the first element of pathnames (host, version and device). The returned values's name, type and version are taken from the last element of pathnames. The intervening elements are used only for their pathname-directory values.

Examples:
  (merge-pathnames-as-file #P"foo/" #P"bar.txt") == #P"foo/bar.txt"

  (merge-pathnames-as-file #P"foo/" #P"./bar.txt") == #P"foo/./bar.txt"

  (merge-pathnames-as-file #P"foo/" #P"/bar/README") == #P"/bar/README"

  (merge-pathnames-as-file #P"/foo/" #P"/bar/" #P'quux/file.txt) == #P"/bar/quux/file.txt"


[Function]
pathname-as-directory pathspec => pathname


Converts the non-wild pathname designator pathspec to directory form, i.e. it returns a pathname which would return a true value if fed to DIRECTORY-PATHNAME-P.


[Function]
pathname-as-file pathspec => pathname


Converts the non-wild pathname designator pathspec to file form, i.e. it returns a pathname which would return a NIL value if fed to DIRECTORY-PATHNAME-P.


[Function]
pathname-directory-pathname pathname => result

Returns a complete pathname representing the directory of pathname. If pathname is already a directory pathname (name nil, type nil) returns a pathname equal (as per pathname-equal) to it.


[Function]
pathname-parent-directory pathname => result

Returns a pathname which would, by name at least, contain pathname as one of its direct children. Symlinks can make the parent/child relationship a like opaque, but generally speaking the value returned by this function is a directory name which contains pathname.

The root directory, #P"/", is its own parent. The parent directory of a filename is the parent of the filename's dirname.

Traversing directories


[Function]
list-directory dirname &key follow-symlinks => list


Returns a fresh list of pathnames corresponding to all files within the directory named by the non-wild pathname designator dirname. The pathnames of sub-directories are returned in directory form - see PATHNAME-AS-DIRECTORY.

If follow-symlinks is true (which is the default), then the returned list contains truenames (symlinks will be resolved) which essentially means that it might also return files from outside the directory. This works on all platforms.

When follow-symlinks is NIL, it should return the actual directory contents, which might include symlinks. (This is currently implemented only on SBCL and CCL.)


[Function]
walk-directory dirname fn &key directories if-does-not-exist test follow-symlinks => |


Recursively applies the function designated by the function designator fn to all files within the directory named by the non-wild pathname designator dirname and all of its sub-directories. fn will only be applied to files for which the function test returns a true value. (The default value for test always returns true.) If directories is not NIL, fn and test are applied to directories as well. If directories is :DEPTH-FIRST, fn will be applied to the directory's contents first. If directories is :BREADTH-FIRST and test returns NIL, the directory's content will be skipped. if-does-not-exist must be one of :ERROR or :IGNORE where :ERROR (the default) means that an error will be signaled if the directory dirname does not exist.

If follow-symlinks is true (which is the default), then your callback will receive truenames. Otherwise you should get the actual directory contents, which might include symlinks. This might not be supported on all platforms. See LIST-DIRECTORY.

Temporary Files

Synopsis

Create a temporary file and return its name:

CL-USER> (temporary-file:with-output-to-temporary-file (foo)
           (print "hello" foo))
#P"/var/folders/Yu/YuNMNBNPGoqs9G-1Wmj1dk+++TI/-Tmp-/temp-yjck024x"

Create a temporary file, read and write it, have it be deleted automatically:

CL-USER> (temporary-file:with-open-temporary-file (foo :direction :io)
           (print "hello" foo)
           (file-position foo 0)
           (read foo))
"hello"

Default temporary file directory
By default, temporary files are created in a system specific directory that defaults based on operating system conventions. On Unix and Unix-like systems, the directory /tmp/ is used by default. It can be overridden by setting the TMPDIR environment variable. On Windows, the value of the environment variable TEMP is used. If it is not set, temporary file creation will fail.
Defining the temporary file directory

The Lisp application can set the default directory in which temporary files are created by the way of the temporary-files logical pathname host:

(setf (logical-pathname-translations "temporary-files") '(("*.*.*" "/var/tmp/")))
This would set the directory for temporary files to /var/tmp/. For more information about logical pathnames, please refer to Common Lisp the Language, 2nd Edition and the Common Lisp HyperSpec.

Physical path names have restrictions regarding the permitted character in file names. If these restrictions conflict with your desired naming scheme, you can pass a physical pathname as TEMPLATE parameter to the temporary file generation function.

Here are a few examples:

CL-USER> (logical-pathname-translations "temporary-files")
(("*.*.*" #P"/var/folders/Yu/YuNMNBNPGoqs9G-1Wmj1dk+++TI/-Tmp-/"))
CL-USER> (temporary-file:with-open-temporary-file (foo)
           (pathname foo))
#P"/var/folders/Yu/YuNMNBNPGoqs9G-1Wmj1dk+++TI/-Tmp-/temp-6rdqdkd1"
This used the temporary directory established in the TMPDIR environment variable, by the way of the definition of the temporary-files logical host definition.
CL-USER> (temporary-file:with-open-temporary-file (foo :template "/tmp/file.with.dots.in.name.%.txt")
           (pathname foo))
#P"/tmp/file.with.dots.in.name.2EF04KUJ.txt"
Here, a physical pathname was used for the :template keyword argument so that a filename containing multiple dots could be generated.
CL-USER> (temporary-file:with-open-temporary-file (foo :template "temporary-files:blah-%.txt")
           (pathname foo))
#P"/var/folders/Yu/YuNMNBNPGoqs9G-1Wmj1dk+++TI/-Tmp-/blah-72mj450d.txt"
This used the temporary-files logical pathname host, but changed the filename slightly.
CL-USER> *default-pathname-defaults*
#P"/Users/hans/"
CL-USER> (temporary-file:with-open-temporary-file (foo :template "blah-%.txt")
           (pathname foo))
#P"/Users/hans/blah-5OEJELG2.txt"
Here, a relative pathname was used in the template, which caused the file to be generated in the directory established by *default-pathname-defaults*.

Alternatively, the *default-template* special variable can be set to define a custom default template for generating names.

Security
The TEMPORARY-FILE library does not directly address security issues. The application that uses it needs to take additional measures if it is important that files created by one process cannot be accessed by other, unrelated processes. This can be done by using the system dependent security mechanisms like default file permissions or access control lists.
Dictionary

[Function]
open-temporary &rest open-arguments &key template generate-random-string max-tries &allow-other-keys => stream

Create a file with a randomly generated name and return the opened stream. The resulting pathname is generated from template, which is a string representing a pathname template. A percent sign (%) in that string is replaced by a randomly generated string to make the filename unique. The default for template places temporary files in the temporary-files logical pathname host, which is automatically set up in a system specific manner. The file name generated from template is merged with *default-pathname-defaults*, so random pathnames relative to that directory can be generated by not specifying a directory in template.

generate-random-string can be passed to override the default function that generates the random name component. It should return a random string consisting of characters that are permitted in a pathname (logical or physical, depending on template).

The name of the temporary file can be accessed calling the pathname function on stream. For convenience, the temporary file is opened on the physical pathname, i.e. if the template designate a logical pathname the translation to a physical pathname is performed before opening the stream.

In order to create a unique file name, open-temporary may loop internally up to max-tries times before giving up and signalling a cannot-create-temporary-file condition.

Any unrecognized keyword arguments are passed to the call to open.

[Macro]
with-output-to-temporary-file (stream &rest args) &body body => pathname

Create a temporary file using open-temporary with args and run body with stream bound to the temporary file stream. Returns the pathname of the file that has been created. See open-temporary for permitted options.

[Macro]
with-open-temporary-file (stream &rest args &key keep &allow-other-keys) &body body => values

Create a temporary file using open-temporary with args and run body with stream bound to the temporary file stream. Returns the values returned by body. By default, the file is deleted when body is exited. If a true value is passed in keep, the file is not deleted when the body is exited. See open-temporary for more permitted options.

[Special variable]
*default-template*

This variable can be set to a string representing the desired default template for temporary file name generation. See open-temporary for a description of the template string format.

[Condition type]
cannot-create-temporary-file

Signalled when an attempt to create unique temporary file name failed after the established number of retries.

[Condition type]
invalid-temporary-pathname-template

Signalled when the template argument to open-temporary does not contain a valid template string. The template string must contain a percent sign, which is replaced by the generated random string to yield the filename.

[Condition type]
missing-temp-environment-variable

(Windows only) Signalled when the TEMP environment variable is not set.

[Logical Pathname Host]
temporary-files

This logical pathname host defines where temporary files are stored by default. It is initialized in a suitable system specific fashion: On Unix and Unix-like systems, the directory specified in the TMPDIR environment variable is used. If that variable is not set, /tmp is used as the default. On Windows, the directory specified in the TEMP environment variable is used. If it is not set, a missing-temp-environment-variable error is signalled.

Modifying the file system


[Function]
copy-file from to &key overwrite => |


Copies the file designated by the non-wild pathname designator from to the file designated by the non-wild pathname designator to. If overwrite is true (the default is NIL) overwrites the file designtated by to if it exists.


[Function]
copy-stream from to &optional checkp => |


Copies into to (a stream) from from (also a stream) until the end of from is reached. The streams should have the same element type unless they are bivalent. If checkp is true (which is the default), the function will signal an error if the element types aren't the same.


[Function]
delete-directory-and-files dirname&key if-does-not-exist => |


Recursively deletes all files and directories within the directory designated by the non-wild pathname designator dirname including dirname itself. if-does-not-exist must be one of :ERROR or :IGNORE where :ERROR (the default) means that an error will be signaled if the directory dirname does not exist.

Warning: this function might remove files from outside the directory, if the directory that you are deleting contains links to external files. This is currently fixed for SBCL and CCL.

The PATH package


[Package]
(defpackage path)

Provides a set of short names for commonly used pathname manipulation functions (these are all functions from the cl-fad package which are being exported under different names):
dirname
pathname-as-directory
basename
cl:file-namestring
-e
file-exists-p
-d
directory-exists-p
catfile
merge-pathnames-as-file
catdir
merge-pathnames-as-directory
rm-r
delete-directory-and-files
=
pathname-equal
absolute-p
pathname-absolute-p
relative-p
pathname-relative-p
root-p
pathname-root-p

 

Acknowledgements

The original code for this library was written by Peter Seibel for his book Practical Common Lisp. I added some stuff and made sure it worked properly on Windows, specifically with CCL. Thanks to James Bielman, Maciek Pasternacki, Jack D. Unrue, Gary King, and Douglas Crosher who sent patches for OpenMCL, ECL, ABCL, MCL, and Scieneer CL.

$Header: /usr/local/cvsrep/cl-fad/doc/index.html,v 1.33 2009/09/30 14:23:12 edi Exp $

BACK TO MY HOMEPAGE