Over the last months, I’ve started several hacks on Scheme48. My interest in Scheme48 was initially sparked by Jivera, who started to bring a Pika-style C API to Scheme48. The features/projects I’ve been working on:

• The new buildsystem: it is now easily possible to build Scheme48 completely from scratch, bootstrapping it with another Scheme48 installation.
• R5RS-compatible keywords (e.g. #:foo). There seems to be some controversy about the usefulness and “good taste” of keywords, but I need them for my GOOPS-compatible TinyCLOS version.
• Relaxed string escaping, i.e. allow e.g. (display “Hello World!
  “).
• A scheme48 executable that allows command-line access to the exec language; you can do

  scheme48 -cl packages.scm -o mystructure
  

  which is equivalent to typing:

  ,config ,load packages.scm
  ,open mystructure
  

  at the Scheme48 prompt.
• Some initial work on a PLTish FFI. See Foreign Interface for PLT Scheme for how this FFI looks like. The cool thing about it is that you don’t have to delve into C for creating bindings anymore.
• Porting Kali Scheme. This has just started; so far, I’ve been able to get the Kali VM to build on top of Scheme48 1.2, but it is probably far from working.

Most of the above is still not polished and not ready for release or even sending patches upstream. I have, however, submitted some very simple patches to the Scheme48 mailing list, and got no response from the maintainers. Should working with them not work out, I might have to create my own fork of Scheme48. In fact I have forked already, but if I can’t get (parts) of my stuff into mainline Scheme48, I might start to actively push my fork as “extended Scheme48″ or with a new name.

About two months ago, I started thinking about a highlevel, implementation-independent networking API for Scheme. Riastradh’s TCP proposal is well thought-out, but I was thinking of something that would extend naturally to other protocols such as Unix domain sockets, or SSL/TLS. I took the TCP proposal, generalized and extended it a bit and. The result is online, I am appreciating comments! I currently don’t have the time to polish this up to SRFI-ready quality and submit it, but hopefully will do so next year. Implementation Status After harvesting some feedback from the #scheme community, I started to implement my proposal for Scheme 48 originally as patch against the development version, but since that is unfortunatly not publically available, I decided to split it off and make it work with vanilla Scheme 48 1.3. Currently, the implementation covers almost the whole TCP part, modulo some holes to fill out. For the curious and brave, there is a snapshot tarball of the code is linked from the CSW - be aware that this is in half-finished state.

I agree with madduck when he writes that the way the law deals with drugs is very much black-and white and should be diversified. Quoting him:

… In particular, the law should differentiate between * drugs that are dangerous to the taker, or can cause long-term damages, such as physical or mental addictions. This class would be primarily made up of the so-called hard drugs: opiates, entactogens, inhalants, steroids, sedatives, and stimulants.
* drugs that cause hightened awareness and give the taker the ability to approach life, society, or any sort of philosophical question from different angles. In places, these are referred to as intellectual drugs: hallucinogens and (some) herbals.

I think the above text fragment perpetuates a common mis-use of the word “drug” meaning “illegal drug”. Certainly Alcohol and tobacco fall under the first category, as both not only “can cause” long-term damages, but certainly will do so if used in “appropriate” dose and frequency. Both are legally and socially tolerated drugs, however, at least here in Austria. The “danger” of any drug will vary with dose and frequency of (ab-)use, see also Wikipedia. Personally, what bugs me more than the laws, is the way this issue is handled by the mass media. Often enough, one hears about “drug deaths” or somone being busted with this-and-that amount of “drugs”, not even mentioning what kind of drugs were involved, which makes people with a basic knowledge of the topic think “bah, what crappy journalism” and probably deepens the black-and-white world-view of the others. When the broad public is constantly exposed to such an undifferentiated view, this will certainly shape public opinion, making more liberal (or rather libertarian) legislation on that topic unlikely.

Today, I implemented open-tcp-listener-accept-loop from Riastradh’s TCP proposal, and extended my own networking proposal to reflect this. For quick testing, I wrote a simple server that evaluates scheme code. No sandboxing or other fancies are implemented, but I think it shows pretty nicely how easy it is to build a multi-threaded TCP server in Scheme48. The code is available via lisppaste. And to all of you that are like me still in 2005: An guatn Rutsch, und a guats neichs!

We all know a long time that CVS sucks; the author of that article (in 2002) listed his main gripes with CVS: missing file renaming and not being able to remove directories. So far so I agree completly, these two things make CVS annoying to use, and everyone, even casual CVS users, will stumble upon these mis(sing) features. At that point, both subversion and what later became GNU Arch (to my knowledge, the first Free Software, distributed RCS) were in pre-1.0 versions, but becoming ready for use by the braver souls. While subversion has certainly fixed the renaming and directory problems, subversion has a fundamental shortcoming, making it an obstacle for FOSS to reach its full potential: it restrains the developer that may come along and want to experiment with the code. And no, it is not enough that the developer can checkout the latest code and submit a patch to the mailing list. This might work on projects where such patches are both trivial and quickly fed into the repository. But for more serious work, you will want to keep track of your work for yourself, either so you can review and alter the steps (changesets) done later on, or since you have to submit it piecemeal anyway. It could also happen that another, non-privileged developer wants to work on a thing that needs your modifications, before your patch has been approved and committed, or even before you are fully done with polishing it up fully for inclusion into the mainline. This other developer may also be just interested in your work and like to test it, and while doing so, he finds a little bug and wants to fix it. All this does happen in the real world, even on a large scale, e.g. with Linux distributions, and it’s called branching. The problem with non-distributed RCSs is: you need write access to the repository to create a branch. This is both true in CVS and subversion (although in CVS, branching makes little sense, as merging is a real PITA). To get write access, you have to go through whatever hurdles there may be in that particuliar project until you are deemed worthy of commit access. It is of course clear why it has to be that way: would the developers of that project grant everyone and their dog commit access, they would soon be busy with undoing the damage done by commits that don’t meet quality (or other) criteria, and not having any time left for hacking. This is not a problem with a distributed RCS, however - you simply have (typically) one person maintaining the “official” version of the source code, typically that’s the same person that rolls release tarballs. For example with Linux, it is Linus (for the development branch). That person can have their own repository, where everyone else only has read access, and selectively pulls changesets (patches) from repositories of peer developers. I hope I was able to made clear why using non-distributed RCS for a FOSS project can restrain the possibilities, limiting indivual freedom. So, if you believe in freedom (and software freedom in particuliar), consider giving a distributed RCS of your choice a try. I can heartily recommend darcs, (altough I’m not sure how it scales to bigger projects) but there are several alternatives out there. And in case you want to migrate an existing project, or give your users and developers a choice of RCS, you can use tailor to migrate your old repository, or even establish a two-way gateway between two RCS.

I couldn’t resist the temptation, and started to write yet another Scheme logging library; this has been on my TODO-list for quite some time now, but now it became more urgent, since we need a logging facility for conjure. I hope jao will be pleased to see that my branch of conjure already makes use of the new logging module . Of course, I didn’t in fact start from scratch, but had a look at the various logging libraries available; the Scheme ones being:

• The logging package in guile-lib. I couldn’t really use that code, since it’s uses GOOPS, Guile’s CLOS-like object system. Also it doesn’t support hierarchies of loggers, which I consider an important feature, since it allows flexible configuration of the logging behaviour for a software system that is contains of multiple subsystems. For example, you may want to enable debugging output in a specific subsytem, while only getting warnings and errors for everything else.
• log4scm. I don’t know what to say about that but: strange. While named in the log4X tradition initiated by log4j, a rather powerful (and complex, IMHO) logging system for Java, it seems very unflexible, and not suitable for a general-purpose logging system. Nevertheless it uses threads, a callback system for fatal logs, and SSAX to parse XML files for configuration - speak about heavyweight! Strange license, too. I will refrain from commenting further .

I also had a look at log4r and the Python logging package; both seem worthy taking as a starting point. What I’ve come up with so far has a pretty similiar model to these systems; you send a log entry that should be logged to a logger, which has zero or more handlers that are responsible for actually doing anything with the log entry, such as writing it to a port. All loggers, besides the root logger have a parent, (also a logger) which log entries are sent to, in addition to being processed by the handlers attached to the logger; the propagation to the parent can also be turned off. The logger hierarchy is established by a hierarchical namespace: The logger with the name '(conjure cc) is a child of the logger named '(conjure). This may all sound overly complicated, but getting started is very straightforward:

;; ,open spells.logging
(define log-info (make-log ‘(test) ‘info))
(configure-logger ‘())
(log-info “hello world!”)

make-log creates a logging procedure. Logging procedures take a single argument to be logged; the argument may be an arbitrary object; it is up to the handlers on how to go about formatting this object. The argument passed to the logging procedure is wrapped in a log entry, which additionally contains the log level. The name of the logger that is to be used is '(test), and 'info is the log level. If the logger '(test) doesn’t already exist, it is created on the fly with no handlers and the root logger as a parent; it can be configured later. This separation of logger creation and configuration is important: In a real application, the logging can be configured from the command line or a configuration file; you don’t want to (hard-)code it into the same module that uses the logger. (configure-logger '()) configures the root logger (which has the empty list as name) with a default configuration. configure-logger also has an optional argument in which you can provide a custom configuration, but I won’t go into this now, since this part of interface is most likely to change. The default configuration amounts to logging everything on (current-output-port). Finally, we log a simple string through the logging procedure log-info. The output looks like this:

(test: [info] hello world!)
Well, enough of this for now; if you got curious you can take peek at the code.

Mike Sperber finally reviewed my patches; they unfortunatly were not deemed appropriate for the Scheme48 distribution, see this thread. Hopefully these issues can be sorted out in the not-to-far future. If you also need the one or other features from my hacks so far, and are adventurous, you can conact me on #scheme on freenode for help with getting and building my fork. Writing portable Scheme programs with scmxlate and spells For the conjure project, we are using scmxlate to make it portable across multiple Schemes. For that purpose, I slightly extended scmxlate and built a kind of meta-module system on top of it. Here is how that looks like: You write a description of the modules your application or library is made of; this is done in a sightly extended subset of the Scheme48 configuration language. For example (from this page, see there for more details):

 (define-structure conjure.foo.bar (export qux baz
					  ((froboz fribnitz)
					  :syntax))
  (open scheme srfi-1 srfi-13 conjure.util.files)
  (dialect (guile (open ice-9.syncase))
           (scheme48 (open extended-ports)))
  (files (foo basic-bar)
         (foo bar)))

This defines the module conjure.foo.bar, which exports the identifiers qux and baz, as well as the macros froboz and fribnitz. The open clause means: this module is written in R5RS Scheme (scheme), plus additionally using srfi-1, srfi-13 and the conjure.util.files module. The dialect clause can be used for specifiying additional modules needed for the dialect-specific code for the module. Finally, the files clause tells us that this module consists of the files foo/basic-bar.scm and foo/bar.scm. This file then gets processed by scmxlate, generating a file with the right module definitions for the selected target dialect, e.g.:

(define-module (conjure foo bar)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-13)
  #:use-module (conjure util files)
  #:use-module (ice-9 syncase)
	
   #:export (qux baz)
  #:export-syntax (froboz fribnitz))
	
 ;; code for foo/basic-bar.scm and foo/bar.scm follows

will be generated for Guile. We recently split scmxlate and the meta-module systeem out of conjure, see the scmxlate category in the conjure archive, so it is available for other projects. Now the split-out of the general purpose modules into a separate project — SPELLS, (S)pell’s a (P)ortability (E)nvironment (L)ibrary for (L)ots of (S)chemes — is in preparation. This library will further aid writing portable code, by providing a consistent interface to functionality that has different APIs in different Schemes. These modules are already there:

• spells.util.pregexp: Dorai Sitaram’s portable regular expressions
• spells.util.named-args: Conviniently create procedures with named arguments
• spells.util.table: Hash tables
• spells.util.condition: SRFI 34 and SRFI 35
• spells.system.file: Lots of file-related proedures, e.g. copy-file, directory?, …

All of those are implemented for all of mzscheme, Guile, Scheme48 and Gauche, using native functionality where available. Other modules will surely follow, a project has been requested at Gna! to facilitate contribution. The spells library will also be used by my upcoming port of G-Wrap to Scheme48.

I’ve decided to laying the foundation for creating bindings of the GNOME software stack for Scheme48, specifically: create bindings for GLib/GObject. At the core of GNOME, there is the GLib Object System (GObject), on top of which GTK+ and the rest of the developer platform are built. Thanks to the header scanning tool h2def.py, which creates a S-Expression representation of the API, you are able to create bindings for any GObject-based library, once you got the core bindings right. I have a rough plan how to do this; basically I plan to fork guile-gnome and reuse the existing infrastructure (e.g. .defs processing). The Guile-GNOME fork will drive a G-Wrap that’s capable of emitting code for the upcoming Scheme48 FFI. Work on G-Wrap/Scheme48 has already progressed to a degree where it’s possible to turn to the GObject bindings and fill out the holes as needed in the process. The plan is also to make the resulting bindings “mostly compatible”, so that programs written for Guile-GNOME will also work with Scheme48-GNOME and vice versa. Since Scheme48 has no built-in object system, and the object system would have to be compatible with GOOPS (Guile’s object system) anyway to achieve source-compatibility between the bindings, I started working on a GOOPS-compatible version of TinyCLOS.

This are some random thoughts I posted to the G-Wrap development mailing list regarding creating bindings for C libraries for Scheme. Although partly G-Wrap specific, it also might be of interest to other Scheme hackers. I’m not at all convinced my effort to make G-Wrap support Scheme 48
makes much sense at all, for these reasons:

• Most of G-Wraps complexity stems from C code generation, which is
  not necessary with a Scheme-level FFI, such as Scheme 48 and PLT
  Scheme provide. It seems a different, more simple tool, with the
  different features would be much more suitable: With a Scheme-level FFI, you can conviniently bind a given C API
  by writing Scheme code; it seems backwards to use a code
  generator, as you have one in Scheme itself in the form of
  macros. The problem is a different one: extracting certain
  information from the C API that is not (convieniently) available
  without invoking a compiler, such as the values of C macros and
  enumerations. With the code-generation approach, this knowledge is
  compiled into wrapper functions (and thus hidden away). With a
  tool I envision, that knowledge would be _extracted_ at the tools
  run time, by generating simple C programs that print those values
  in S-expression form, invoking the compiler on the C code, and
  running the resulting executable. This is quite different from
  what G-Wrap does, and retrofitting such functionality into G-Wrap
  would make it even more complex. I can really recommend reading the paper about the PLT FFI, linked
  from the CSW, to see how expressive a Scheme-Level FFI is; hiding it
  below a code generator only takes away from that.
• Now that my focus has shifted from Guile to Scheme 48, and
  portable Scheme in general, I view my rewrite using GOOPS
  ambivalently; the rewrite has certainly made G-Wrap more flexible
  and extensible, and probably given it a clearer structure,
  however, due to GOOPS’ use of keywords (while they are
  R5RS-compatible and supported by some other implementations),
  GOOPS is fundamentally non-standard, without touching the core of
  implementations that don’t provide #:foo-style keywords it isn’t
  possible to implement it on other Schemes. IMO, G-Wrap supporting
  multiple back-ends only makes sense when it also is able to run on
  these implementations, or else it probably won’t get much usage.

So in short: G-Wrap’s future looks brightest when staying a Guile
(-only) tool; a much more simple, portable C-binding tool can be
achieved by:

• Implementing a PLT-style FFI. This already goes a long way towards
  easing the work of creating bindings; I wrote a simple mysql
  binding for Scheme 48 in 135 lines of pure Scheme. Actually, the
  RTI (run time information) layer in G-Wrap can be seen as a
  half-implementation of that approach; implementing a subset of the
  PLT FFI for Guile based on that part of G-Wrap would be not very
  hard (speaking from my experience on doing the same thing for
  Scheme 48).
• Supplement that with tools that can extract the low-level C API
  information from headers, by parsing and compilation (as outlined
  above). Also G-Wrap could profit from a C header parser, as
  writing the declarations for the wrappers is quite tedious if more
  than a few functions are involved. I haven’t really looked at it,
  but scheffig is a tool that looks similiar to G-Wrap on a
  superficial level (generates C wrapper functions) and leverages
  the FCC C parser written by Mike Sperber, which probably is quite
  portable.
• The low-level API information, as extracted in the above step still
  doesn’t contain enough information for useful Scheme bindings, as
  part of the information needed for binding a C library are only
  available in its documentation, for example the semantics of memory
  allocation, for example whether a “double *” function argument in
  fact an array of a certain size, specified by another argument of
  that function, or a output argument. This information would be
  encapsulated in hand-written high-level bindings that make use of
  the automatically generated low-level bindings that result from
  header parsing.

I began to program in Scheme using the Guile interpreter. While I knew that its not blazingly fast, its the official GNU extension language, and as feeling quite close to the goals of the GNU project and the FSF, I felt confident it was the right choice, also considering the supreme quality of other GNU software, such as the GNU toolchain (GCC, binutils, gas) and the Gimp, to name just a few. So, feeling confident with my choice, I started to crank out code and eventually even took over maintainership over a Guile application, namely G-Wrap, after giving it a complete brush-up code-wise. When Tom Lord, of GNU Arch fame (and coincidently a former Guile maintainer), announced the Pika project (a from-scratch Scheme implementation, with ambitous goals), I decided to get on the mailing list and lurk, and later got opportunities to contribute a bit. Unfortunatly that project stalled due to time constraints on Tom’s side. Anyway it was a very valuable experience, as it opened my eyes to other Scheme implementations, foremost Scheme 48. Looking at its code base was a real eye-opener: as opposed to Guile, it was, except for a tiny amount of glue code, written in Scheme; its virtual machine is written in a subset of Scheme called Pre-Scheme that can be compiled to efficient C code, so you get most of the benefits of Scheme and the speed of C. Around the same time I discovered Scheme 48, I got involved in conjure, a make replacement being written in portable Scheme. I split off the library code from it to form a separate project, spells. As spells is basically a layer above the individual Scheme implementations which hides away their differences and presents their features (foremost the module system, but also hash tables, file system access, …) in a portable way, I gained some experience regarding the different module systems and how they compare, and I honstly must say that Guile’s is basically unusable with macros. Here’s why: A decent module system allows for macro code to refer to bindings in the module they are defined in, without requiring these bindings to be also exported; imagine a macro that expands to a call to some helper procedure that should itself should not be visible from outside the module. A real-life example are the reference implementations of SRFI-34 and SRFI-35, which define the helper procedures guard-aux and type-field-alist->condition, respectively. This deficiency (which is not present in Scheme 48, PLT Scheme, Gauche and probably most other Schemes) is really unfortunate, as it makes modules, when used with macros, a lot less usable. This, besides the total orientation toward interpretation and the codebase that is mostly in C are reasons why I’d recommend Scheme newbies also looking at other options before settling on Guile because it’s the “official GNU extension language”, as I did .

I’m going to get paid for open source developement! Wooha! More specifically, the company I work for since 2 months is going to switch away from CVS as SCM tool. We had a look several SCM systems, among them Subversion, GNU Arch and several commercial offerings, including BitKeeper :-0. The final contestants were GNU Arch and AccuRev. Subversion fell flat for its missing features (mediocre merging support, no distribution), and BitKeeper was not considered seriously because they don’t even list prices at their site (which indicates a fairly high price tag, normally and due to opposition from myself (for the obvious political reasons). Well, finally the case was made for GNU Arch, and since it lacks a decent GUI (currently the GUIish guys here use WinCVS), it was decided to develop the GUI in-house, and as an open source project!. It will probably be developed using Python and GTK+, so it will run also on Windows (which is a requirement). Expect more news, including project announcement soonish.

This are some random thoughts I posted to the G-Wrap development mailing list regarding creating bindings for C libraries for Scheme. Although partly G-Wrap specific, it also might be of interest to other Scheme hackers. I’m not at all convinced my effort to make G-Wrap support Scheme 48
makes much sense at all, for these reasons:

• Most of G-Wraps complexity stems from C code generation, which is
  not necessary with a Scheme-level FFI, such as Scheme 48 and PLT
  Scheme provide. It seems a different, more simple tool, with the
  different features would be much more suitable: With a Scheme-level FFI, you can conviniently bind a given C API
  by writing Scheme code; it seems backwards to use a code
  generator, as you have one in Scheme itself in the form of
  macros. The problem is a different one: extracting certain
  information from the C API that is not (convieniently) available
  without invoking a compiler, such as the values of C macros and
  enumerations. With the code-generation approach, this knowledge is
  compiled into wrapper functions (and thus hidden away). With a
  tool I envision, that knowledge would be _extracted_ at the tools
  run time, by generating simple C programs that print those values
  in S-expression form, invoking the compiler on the C code, and
  running the resulting executable. This is quite different from
  what G-Wrap does, and retrofitting such functionality into G-Wrap
  would make it even more complex. I can really recommend reading the paper about the PLT FFI, linked
  from the CSW, to see how expressive a Scheme-Level FFI is; hiding it
  below a code generator only takes away from that.
• Now that my focus has shifted from Guile to Scheme 48, and
  portable Scheme in general, I view my rewrite using GOOPS
  ambivalently; the rewrite has certainly made G-Wrap more flexible
  and extensible, and probably given it a clearer structure,
  however, due to GOOPS’ use of keywords (while they are
  R5RS-compatible and supported by some other implementations),
  GOOPS is fundamentally non-standard, without touching the core of
  implementations that don’t provide #:foo-style keywords it isn’t
  possible to implement it on other Schemes. IMO, G-Wrap supporting
  multiple back-ends only makes sense when it also is able to run on
  these implementations, or else it probably won’t get much usage.

So in short: G-Wrap’s future looks brightest when staying a Guile
(-only) tool; a much more simple, portable C-binding tool can be
achieved by:

• Implementing a PLT-style FFI. This already goes a long way towards
  easing the work of creating bindings; I wrote a simple mysql
  binding for Scheme 48 in 135 lines of pure Scheme. Actually, the
  RTI (run time information) layer in G-Wrap can be seen as a
  half-implementation of that approach; implementing a subset of the
  PLT FFI for Guile based on that part of G-Wrap would be not very
  hard (speaking from my experience on doing the same thing for
  Scheme 48).
• Supplement that with tools that can extract the low-level C API
  information from headers, by parsing and compilation (as outlined
  above). Also G-Wrap could profit from a C header parser, as
  writing the declarations for the wrappers is quite tedious if more
  than a few functions are involved. I haven’t really looked at it,
  but scheffig is a tool that looks similiar to G-Wrap on a
  superficial level (generates C wrapper functions) and leverages
  the FCC C parser written by Mike Sperber, which probably is quite
  portable.
• The low-level API information, as extracted in the above step still
  doesn’t contain enough information for useful Scheme bindings, as
  part of the information needed for binding a C library are only
  available in its documentation, for example the semantics of memory
  allocation, for example whether a “double *” function argument in
  fact an array of a certain size, specified by another argument of
  that function, or a output argument. This information would be
  encapsulated in hand-written high-level bindings that make use of
  the automatically generated low-level bindings that result from
  header parsing.