MJ Ray: TPM/DRM
There's a protest/rally outside Apple
Store in Regent Street, London, England
from 2pm on Saturday 2006-09-30,
just ahead of 2009-10-03's Day Against DRM.
Note to self: 11:30 PM is not a good time to go home if you need to pass the Oktoberfest on the way.
(Obviously, the "after" image was not suitable for this medium.)
sysvinit package.
Before trying this, make sure your installation is up to date as we’ve had to split out some parts of sysvinit into a new sysvutils package. If you’re up to date, and want to try it out, install the upstart and upstart-compat-sysv packages from universe.
Note that the first reboot after you’ve installed the packages (from sysvinit to upstart) will be a little tricky … use reboot -f.
If your system boots and shuts down normally, everything’s working just fine. Note that both will be somewhat more quiet than you’re used to, unless you have usplash running.
Throughout the rest of this entry, I’ll try to answer some of the questions and comments that I’ve received since the last post.
Events
As I talked about previously, upstart is an event-based init daemon. Events are the primary force for having your services and tasks started and stopped at the appropriate time and in the appropriate order.
So what are events and where do they come from? (Note that this part is under development, so may change in later releases).
Events are just simple strings that may be sent by any process when something it is tracking the state of changes. They have no state or longevity, and if, when queued, they do not cause any job state changes, then they have no effect unless they are sent again.
Jobs can list which events cause them to be started if they are not already running and which events cause them to be stopped if they are running. Multiple start and stop events may be listed, in which case the first to occur changes the job until the next one occurs.
upstart itself generates the following system events:
shutdown tool included in the package also causes one of the following events to be sent once the “shutdown” event has been handled:
shutdown -e event.initctl trigger EVENT. For now this is just the event string, however it’s intended that the event may include other details including environment variables and even file descriptors.
Typical example
To clarify how it all hangs together, here’s an example (using fictional names) of how the tasks and events can be arranged to provide race-free mounting of filesystems.
/etc/fstab/etc/fstab and if all are mounted, generates the “writable-filesystem” event./etc/fstab will be checked and mounted.
The only reason they wouldn’t be is if there’s an error of some kind, and that means you have larger problems anyway and the system administrator would have a shell to fix it. Of course, the moment they finish checking the filesystem and mount it, the boot process would carry on.
There’s no reason that any of these events need to be generated by the upstart daemon itself, it can receive them from any other daemon on the system such as udev, acpid, etc. This keeps the focus of the init daemon narrow.
A large part of the future development will be working out exactly what kinds of events we want init itself to generate, what kinds we want to come from elsewhere, and what the contents of an event can be.
Getting Involved
If you want to get involved with trying to nudge the direction of upstart development, you can join the upstart-devel mailing list at http://lists.netsplit.com/
Or if you just want to grab the source code, tarballs are published at http://people.ubuntu.com/\scott/software/upstart/ and the bzr archive is at http://bazaar.launchpad.net/\keybuk/upstart/main
sysvinit. It separates jobs into different “run levels” and can either run a job when particular run levels are entered (e.g. /etc/init.d/rc 2) or continually during a particular run level (e.g. /sbin/getty).
The /etc/init.d/rc script is also based on the System V one (and is in the sysv-rc package), it simply executes the stop then start scripts found in /etc/rcN.d (where N is the run level) in numerical order.
Why change it?
Running a fixed set of scripts, one after the other, in a particular order has served us reasonably well until now. However as Linux has got better and better at dealing with modern computing (arguably Linux’s removable device support is better than Windows’ now) this approach has begun to have problems.
The old approach works as long as you can guarantee when in the boot sequence things are available, so you can place your init script after that point and know that it will work. Typical ordering requirements are:
/etc/fstab./etc/fstab may require tools in /usr which is a network filesystem that cannot be mounted until after networking has been brought up.
The two states shown in red (“waiting” and “running”) are rest states, normally we expect the job to remain in these states until an event comes in, at which point we need to take actual to get the job into the next state.
The other states are temporary states; these allow a job to run shell script to prepare for the job itself to be run (“starting”) and clean up afterwards (“stopping”). For services that should be respawned if they terminate before an event that stops them is received, they may run shell script before the process is started again (“respawning”).
Jobs leave a state because the process associated with them terminates (or gets killed) and move to the next appropriate state, following the green arrow if the job is to be started or the red arrow if it is to be stopped. When a script returns a non-zero exit status, or is killed, the job will always be stoped. When the main process terminates and the job should not be respawned, the job will also always be stopped.
As already covered, events generated by the init daemon or received from other processes cause jobs to be started or stopped; also manual requests to start or stop a job may be received.
The communication between the init daemon and other processes is bi-directional, so the status of jobs may be queries and even changes of state to all jobs be received.
How does it differ from launchd?
launchd is the replacement init system used in MacOS X developed as an “Open Source” project by Apple. For much of its life so far, the licence has actually been entirely non-free and thus it has only become recently interesting with the licence change.
Much of the goal of both systems appears initially to be the same; they both start jobs based on system events, however the launchd system severly limits the events to only the following:
/etc/fstab) is available and writable.
The launchd model expects the job to “sit and wait” if it is unable to start, rather than provide a mechanism for the job to only be started when it doesn’t need to wait. Jobs that need /usr to be mounted would need to spin in a loop waiting for /usr to be available before continuing (or use a file in a tmpfs to indicate it’s available, and use that modification as the event).
This is not especially surprising given that Apple have a high degree of control over both their hardware and the actual underlying operating system; they don’t need to deal with the wide array of different configurations that we have in the Linux world.
Had the licence been sufficiently free at the point we began development of our own system, we would probably have extended launchd rather than implement our own. At the point Apple changed the licence, our own system was already more suitable for our purposes.
How does it differ from initng?
Initng by Jimmy Wennlund is another replacement init daemon intended to replace the sysvinit system used by Linux. It is a dependency-based system, where upstart is an event-based system.
The notion of a dependency-based system is interesting to talk about at this point. Jobs declare dependencies on other jobs that need to happen before the job itself can be started. Starting the job causes its dependencies to be started first, and their dependencies, and so on. When jobs are stopped, if running jobs have no dependencies, they themselves can be stopped.
It’s a neat solution to the problem of ordering a fixed boot sequence and the problem of keeping the number of running processes to a minimum needed.
However this means that you need to have goals in mind when you boot the system, you need to have decided that you want gdm to be started in order for it, and its dependencies, to be started. Initng uses run levels to ensure this happens, where a run level is a list of goal jobs that should be running in that run level.
It’s also not clear how the dependencies interact with the different types of job, a dependency on Apache would need the daemon to be running where a dependency on “checkroot” would need the script to have finished running. Upstart handles this by using different events (“apache running” vs. “checkroot stopping”).
Again while interesting, Initng does not solve the problems that we wanted to solve. It can reorder a fixed set of jobs, but cannot dynamically determine the set of jobs needed for that particular boot.
A typical example would be that if the only dependency on the job that configures networking is the mount network filesystems job, then should that job fail or notbe a goal (e.g. because there are no network filesystems to be mounted) the result is that network devices themselves will not be configured. You could make everything a goal, and just use the dependencies to determine the order, however this is less efficient than just ordering the existing sysv-rc scripts (which
can be done at install time).
Another example is that often you simply don’t know whether something is a dependency or not without reading other configuration, for example the mount network filesystems may be a dependency of everything under /usr or may just be a dependency of anything allowing the user to login if it just mounts /home.
The difference in model can be summed up as “initng starts with a list of goals and works out how to get there, upstart starts with nothing and finds out where it gets to.”
How does it differ from Solaris SMF?
SMF is another approach to replacing init developed by Sun for the Solaris operating system. Like initng it’s a dependency-based system, so see above for the differences between those systems and upstart.
SMF’s main focus is serive management; making sure that once services are running, they stay running, and allowing the system administrator to query and modify the states of jobs on the system.
Upstart provides the same set of functionality in this regard, services are respawned when they fail and system administrators can at any time query the state of running services and adjust the state to their liking.
Will it replace cron, inetd, etc?
The goal of upstart is to replace those daemons, so that there is only one place (/etc/event.d) where system administrators need to configure when and how jobs should be run.
In fact, the goal is that upstart should also replace the “run event scripts” functionality of any daemon on the system. Daemons such as acpid, apmd and Network Manager would send events to init instead of running scripts themselves with their own perculiar configuration and semantics.
A system administrator who only wanted a particular daemon to be run while the computer was on AC power would simply need to edit /etc/event.d/daemon and change “on startup” to “on ac power”.
What about compatibility?
There’s a lot of systems administrators out there who have learned how Linux works already and will not want to learn again immediately, there’s also a large number of books that cover the existing software and won’t cover upstart for at least a couple of years.
For this reason, compatibility is very important. upstart will continue to run the existing scripts for the forseeable future so that packages will not need to be updated until the author wants.
Compatibility command-line tools that behave like their existing equivalents will also be implemented, a system administrator would never need to know that crontab -e is actually changing upstart jobs.
Does it use D-BUS?
“To D-BUS people, every problem seems like a D-BUS problem.” —Erik TroanThe UNIX philosophy is that something should do just one job, and do it very well. upstart’s one job is starting, supervising and stopping other jobs; D-BUS’s one job is passing messages between other jobs. D-BUS does provide a mechanism for services to be activated when the first message is sent to them, thereby starting other jobs. Some people have taken this idea and extended it to suggest that all a replacement init system need do is register jobs with D-BUS and turn booting into a simple matter of message parsing. This seems wrong to me, D-BUS would need to be extended to supervise these services, provide means for them to be restarted and stopped; as well as deal with being process #1 which means cleaning up after children whose parent’s have died, etc. It seems far simpler to arrange for D-BUS to send an event to init when it needs a service to be started, and focus on being a very good message passing system. The IPC mechanism used by upstart is not currently D-BUS because of various problems, however it’s always been expected that even if init itself doesn’t communicate with D-BUS directly, there would be a D-BUS proxy that would ensure messages about all init jobs and events would be given to D-BUS and D-BUS clients could send messages to init to query and change the state of jobs. What is the implementation plan? Because this is process #1 we are changing, we want to make sure that we get it right. Therefore instead of releasing a fully-featured daemon and configuration to the world, we’re developing it in the following stages:
/sbin/init while running the existing sysv-rc scripts. This is the shake-down test of the daemon, can it perform the same job as the existing sysvinit daemon without any regressions?/etc/rcS.d scripts replaced by upstart jobs. These consitute the majority of tasks for booting the system into at least single-user mode, and contain many of the current ordering problems and race conditions. If the daemon solves the problems here, it will be a success.upstart package for testing before it becomes the init daemon.
We welcome any experienced users who want to help test this; install the package and follow the instructions in /usr/share/doc/upstart/README.Debian to add a boot option that will use upstart instead of init. If your system boots and shut downs normally (other than a slightly more verbose boot without usplash running) then it is working correctly.
Other types of events will be added as required during development and testing. Currently only a basic client tool (initctl) has been written, compatibility tools such as shutdown will be written over the next week or two before it replaces our sysvinit package.
| LSB 3.x (2006-2008) | LSB 4.x (2008-2010) |
|---|---|
|
Asianux 2.0 Debian 4.0 (”etch”) Mandriva Corporate 4.0 Red Hat Enterprise Linux 4 & 5 SUSE Linux Enterprise 9 & 10 Ubuntu 6.06 LTS (”dapper”) […] |
Asianux 3.0 Debian “etch” + 1 Mandriva Corporate 5.0 Red Hat Enterprise Linux 6 SUSE Linux Enterprise 11 Ubuntu LTS “dapper” + 1 […] |
#lsb on irc.freestandards.org, where the bulk of LSB development happens.
XC-Bribe: EUR 50.00
Now playing: Auld Lang Syne (monophonic on the phone :-)
Deval’s speech was pretty great. A little something for everyone, and genuinely engaging. I attempted to record it on my pocket MP3 player, but apparently I ran out of batteries early on so it’s not worth posting.
Although Deval has been cast as the furthest left of the candidates, I was struck by his extremely moderate position on most issues. (Standardized testing: good, but needs improvement; economics: need to have several bottom lines, including a profit bottom line and a people bottom line; personal responsibility needs to be balanced with societal responsibility; etc..)
Anyway, the crowd loved it:
Apparently there was no experienced political handler present to orchestrate this event. Deval’s charismatic speech was followed by several less important and less eloquent speakers. Usually, you would want to build up from worst to best. In any event, Chris Gabrieli, the second place finisher, has rebounded instantly and was up there committing to work every day until November on getting Deval elected — I won’t be surprised if Gabrieli ends up in a development-czar role in a Patrick administration. The last place finisher (and original frontrunner) Tom Reilly was nowhere to be seen, a fact that I’m sure will be prominent in tomorrow’s reporting on the event. Where’s the unity without the Attorney General?
Everyone’s bracing for a horrid onslaught of fear-mongering campaign ads from the Republicans. My hope is that Massachusetts citizens are a bit wiser than the rest of the country and it will backfire. I won’t hold my breath, however.
(As a side benefit, I finally met Adam Gaffin, universalhubmeister, for the first time, as well as his eight year old daughter Greta. Greta is now famous for rejecting Deval Patrick’s plan for longer school days.)
Next.