Search Results: "will"

1 May 2010

Sean Finney: Nouveau, Gallium3D, and Compiz on Debian

update: 20100503: It's been brought to my attention that there might have been a regression in one of the libraries, mesa, maybe, so that master might not be as reliable a reference point for this guide. so, just in case, here are the commit hashes that were used from the different projects: for the non-git-savvy: git checkout -b mybranch <hash> we now return to your regular broadcasting... A week or two ago I figured it was about time that I took the plung and tried out the latest nouveau drivers from experimental on my laptop. Unfortunately, it meant bidding adieu to compiz (apart from the bling it does actually provide some pretty damned useful features). I thought I'd give it a shot at least for a while on principal, since I've always had to think of the nvidia drivers as a necessary evil, and their lack of proper xrandr support is really, really annoying. Since then I haven't noticed any problems, apart from the following message after every suspend/resume:
[  931.213239] Uhhuh. NMI received for unknown reason 80 on CPU 0.
[  931.213240] You have some hardware problem, likely on the PCI bus.
[  931.213242] Dazed and confused, but trying to continue

No idea what's causing it, but otherwise the system is stable and functional. If anyone knows what can be done about this, I'd appreciate hearing from you. My graphics card, by the way:
01:00.0 VGA compatible controller: nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300] (rev a1)

In the time since then, the nouveau driver has made its way into unstable, and a lot of distros (ubuntu, fedora, ...) are even using it as the default driver. So before going any further with the 3d/compiz stuff, I highly recommend that you get the packaged version working first. This will not only make sure that the basic driver works, but it will also make recovery/rollback a lot easier. Since it's all packaged in the standard repositories, it's not much harder than installing a few packages and editing a few files... so come on give it a try :) Getting the 2d driver working in Debian sid Really, this is so easy it's hardly worth writing up, but i'm running with the -v flag this morning... Make sure you don't have any reference to nvidia in /etc/modules or other places that might cause it to be automatically loaded. Then remove the proprietary glx bindings, which would otherwise interfere with the nouveau driver.
sudo apt-get remove nvidia-glx

Next, you'll need to install a kernel that includes updated drm support. Normally this would require a kernel >= 2.6.33.1, but the debian kernel maintainers have graciously backported this code to the 2.6.32-4 kernel package. note this isn't the default kernel version in testing/unstable at the time of writing so you have to explicitly install it along with the nouveau driver.
sudo apt-get install linux-image-2.6.32-4-amd64 xserver-xorg-video-nouveau

If you are already running this kernel and using the proprietary nvidia drivers, via module-assistant or otherwise, you /might/ need to uninstall them first. On my system I left them installed in the old kernel version but did not install them on the new kernel version. Required xorg configuration I don't believe that nouveau is picked by default if available at the time of writing, so you have to have the following in your xorg.conf:
sudo sh -c "cat > /etc/X11/xorg.conf" << EOF
Section "Device"
    Identifier     "nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300]"
    Driver         "nouveau"
EndSection
EOF

This was all I needed to do, apart from rebooting into the new kernel. For those who don't want to venture outside the packaging system, this is about as far as you can go for the time being. Though really, if you're bothering to read this far along, you're probably more interested in what comes next :) Getting 3d working (without voiding your distro warranty) So I thought I'd give the 3d-accelerated gallium a go. Their wiki page certainly makes one think twice, but at this point I was seriously fiending to get my bling back (well, the compiz scale plugin, anyway) and figured it wouldn't hurt too much to try. Via google and trawling around in various forums I found a number of how-to documents for getting Gallium3d on debian/ubuntu systems, but most of them recommended doing things in a way that wasn't easily reversible and that could totally hose your system in a not-so-hypothetical worst-case scenario. It's pretty hard to roll back a make install done into /usr, for example, and subsequent package upgrades can leave your system even worse off as they will probably clobber half of what you just did, leaving the other half totally broken. And if you have a laptop which requires the gui for networking (i.e. network-manager), then fixing/rolling-back the changes is not entirely trivial, since you'd likely have to re-download the various drm/mesa/xorg packages. So enter the tried and true unix utility stow (apt-get install stow, of course). stow is a utility that lets you install software into arbitrary folders, and then manage the system-wide installation of the software via a farm of symlinks pointing at the installed location. It's probably more well known among the sysadmin crowd than the desktop user crowd, but it's incredibly useful for the task at hand as it will let us easily install multiple copies of different software in a way that's very easy to update and/or roll-back. RTFM stow(1) for more information. So, on to the juicy stuff... Install needed software, remove problematic packages, setup, etc At least on my system, I had to remove libdrm-dev to keep things from accidentally linking against it instead of the updated version that we install later on below. There's probably a way to override this with CFLAGS but I figure this is simpler.
sudo apt-get remove libdrm-dev
sudo apt-get install stow build-essential xorg-dev git-core libtool mesa-common-dev automake autoconf 
mkdir ~/nouveau

Compile a linux-2.6.34 RC kernel
cd ~/nouveau
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
cd linux-2.6
git remote add -f nouveau git://anongit.freedesktop.org/nouveau/linux-2.6
git checkout -b nouveau-master nouveau/master
make menuconfig

Enable device drivers -> staging drivers -> nouveau. Make any other changes you might want to make (I just took defaults for everything else).
make deb-pkg

The linux-firmware-free debian package ships some of the same stuff from the generated linux-firmware-image package. I decided to opt for the upstream firmware, though I'm not sure that this is necessary:
sudo apt-get remove linux-firmware-free
sudo dpkg -i ~/nouveau/linux-image*.deb ~/nouveau/linux-firmware-image*.deb
sudo update-grub

Install a copy of dri2proto
cd ~/nouveau
git clone git://anongit.freedesktop.org/xorg/proto/dri2proto
cd dri2proto
./autogen.sh --prefix=/usr/local/stow/dri2proto-20100428
sudo mkdir -p /usr/local/stow/dri2proto-20100428
sudo make install
sudo stow -d /usr/local/stow dri2proto-20100428

Install a cutting-edge version of libdrm
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/mesa/drm/
cd drm
./autogen.sh --prefix=/usr/local/stow/drm-20100428 --disable-intel --disable-radeon --enable-nouveau-experimental-api
make
sudo mkdir -p /usr/local/stow/drm-20100428
sudo make install
sudo stow  -d /usr/local/stow  drm-20100428

Likewise for mesa
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/mesa/mesa
cd mesa
./autogen.sh --prefix=/usr/local/stow/mesa-20100428  --enable-debug --enable-glx-tls --disable-asm --with-dri-drivers= --enable-gallium-nouveau --disable-gallium-intel --disable-gallium-radeon --disable-gallium-svga --with-state-trackers=glx,dri --with-demos=xdemos,demos,trivial,tests
make
sudo mkdir -p /usr/local/stow/mesa-20100428 
sudo make install
sudo stow -d /usr/local/stow mesa-20100428

And now the xorg driver...
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/nouveau/xf86-video-nouveau/
cd xf86-video-nouveau
./autogen.sh --prefix=/usr/local/stow/xf86-video-nouveau-20100428
make
sudo mkdir -p /usr/local/stow/xf86-video-nouveau-20100428
sudo make install
sudo stow -d /usr/local/stow xf86-video-nouveau-20100428

Configuration Configure xorg to look in alternate location for xorg modules and set up DRI
sudo sh -c "cat > /etc/X11/xorg.conf" << EOF
Section "Files"
    ModulePath "/usr/local/lib/xorg/modules,/usr/lib/xorg/modules"
EndSection
Section "ServerFlags"
    Option "GlxVisuals" "all"
Endsection
Section "Device"
    Identifier     "nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300]"
    Driver         "nouveau"
EndSection
EOF

I've /heard/ that disabling TV out is a necessity on some cards, though I haven't seen this myself:
sudo sh -c "cat > /etc/modprobe.d/nouveau.conf" << EOF
# not entirely sure this is necessary, but heard it helped with vsync
options nouveau tv_disable=1
EOF

Finally, the one thing that can not be done in /usr/local is making the nouveau DRI driver available. From what I could tell the path that mesa uses to find the DRI libraries is hard-coded (corrections welcome). Since you won't already have this file (otherwise, why would you be following along here?), just drop in a symlink to the file in /usr/local:
sudo ln -s /usr/local/lib/dri/nouveau_dri.so /usr/lib/dri/

Finally, reboot, and hope for the best :) Riding the bleeding edge: how to upgrade later on Again, stow really shows its usefullness here. Using drm as an example case, upgrading an individual component would look something like:
cd ~/nouveau/drm
git clean -dfx
git pull
today= date +%Y%m%d 
./autogen.sh --prefix=/usr/local/stow/drm-$today --disable-intel --disable-radeon --enable-nouveau-experimental-api
make
sudo mkdir -p /usr/local/stow/drm-$today
sudo make install
sudo stow -d /usr/local/stow -D drm-20100428
sudo stow -d /usr/local/stow  drm-$today

Note how the old version is still installed, so you can switch components around pretty freely. Abort! Abort!: How to roll back changes In case your system becomes totally unusable, or you've decided that you just want to go back to the safe and cozy world of what's provided by the debian packages, simply do the following: Assuming that you're using the nouveau driver from unstable, this should be all that is necessary. Conclusion Overall impression I am very, very happy with the resulting setup. I was led to believe that this whole thing was much more unstable and unusable than it actually is! While there are certainly some glitches and missing features, I feel comfortable enough using it that I don't have plans to switch back. But if I find myself in a situation where I can't use the new setup, rolling back shouldn't be a problem either. I figure I'm happy with what I have now and it will only get better. What works, at least to a "kinda works" level. What still doesn't work Further reading, references Thanks A couple shoutouts to those who helped me with this on the way: Feedback / Corrections Unforunately, I've had to disable comments for the time being due to overwhelming quantities of spam. So please drop me a line via email or on IRC (seanius on oftc/freenode).

30 April 2010

Biella Coleman: Baby Panda wants YOU to submit to DEBCONF



bad mood destroyer, originally uploaded by the biella.
People s the deadline is fast approaching. Don t worry though, the best ideas come under deadlines and your proposal need not be lengthy. So get those typing hands out and submit a proposal (a few sentences suffices) for a presentation! The annual Debian conference is being held in NYC this year. First time
setting foot in the imperial empire. Are you doing something related to
free software that might be interesting to share, show off or talk about
at Debconf? I bet you are! We are looking for art installations,
performances, movie screenings, maybe you are wanting to host a
roundtable/panel discussion, coordinate a workshop (you dont have to
know the subject deeply to do this!). I know the deadline is soon, but don t let that scare you. The
submission requirement bar is very low, and we are flexible! here is the submission information: [1] ********************* Calling all potential contributors to DebConf10! One more week until
the final submission deadline! The DebConf team is excited to announce that we are now accepting
proposals for contributions to this year s Debian conference! We
invite you to submit proposals for how you would contribute until the
deadline: soon There are many ways you can contribute, you could present a technical
paper, host a panel discussion, put on a tutorial, do a performance,
an art installation, a debate, host a meeting (BoFS, or Birds of a
Feather Session), or other possibilities that you devise. This year we
are also accepting proposals for tracks=E2=80=94a thematic grouping around a
particular subject, and people to coordinate those tracks. If you are
looking for ideas of things that you could contribute, or have ideas
for things that you would like to see happen at DebConf, have a look
at the Contribution Brainstorm[0] page. DebConf talks will be broadcast live on the Internet where possible,
unless otherwise requested. Videos of the talks will be published on
the web along with the presentation slides and papers. =3D=3D Step 1. Submit Proposal =3D=3D Proposals should provide an overview of your proposed contribution,
similar to an abstract, and be no more than 600 words. Since
contributions can cover a range of issues, please describe what your
proposal covers, be it political (Free Software, law, advocacy,
access, etc.), technical (d-i, OpenPGP, etc.), social (Debian
structures and groups). It s okay to hit all three of these. Your
proposal should also include some information on why you re qualified
to present, or coordinate this topic. For example, you can include
links to posts you have made to Debian mailing lists or other material
that indicates your qualifications. Proposals and their abstracts will
be accepted until May 1st, 2010, 23h59 UTC. To submit your proposal, go to Penta[1] and register as an
attendee. Once you are registered, there is a Submit a proposal
link. You will see your proposal on the site. You can choose between
different presentation types. If you are unsure what type you should
use, please read our glossary of definitions[2]. If it s not there for
some reason, or you have questions, please contact us immediately at
mailto:talks@debconf.org. =3D=3D Step 2. Committee Review =3D=3D The review committee for this year will make its decision by May 1st,
2010. All correspondence will be done by email. =3D=3D Step 3. Paper Submission =3D=3D Like last year, we are asking for papers to be submitted along with
the presentations. A full paper is optional, however we strongly
encourage you to submit one if possible as we intend to provide
written information along with transcripts of the sessions for later
viewing. Having written papers in advance will allow us to get
translations done to help non-native-English speakers feel more
comfortable with the topics presented. Papers are due by July 1st,
2010, and should cover the topic in reasonable depth (3 pages A4 text,
plus pictures and diagrams). We will use LaTeX to typeset the
proceedings. Please submit your paper formatted in LaTeX. Should you
be unfamiliar with LaTeX earlier submission in plain text is also
fine. =3D=3D Step 4. Live Presentation =3D=3D Longer presentations may have a break in the middle and should include
workshop items that directly involve the participants. If using slides
or any other presentation, please consider that your audience will
consist of people who use free software, and your choice of
application to prepare and display the slides should reflect this if
at all possible. =3D=3D Fine Print Publication Rights =3D=3D DebConf requires non-exclusive publication rights to papers,
presentations, and any additional handouts or audio/visual materials
used in conjunction with the presentation. The authors have the
freedom to pick a DFSG-free license for the papers themselves and
retain all copyrights. The presentations will be recorded, and may be
broadcast over the Internet. Any copies of the presentation will be
made available =3D=3D Failure to Submit =3D=3D In the event that a deadline is missed we reserve the right to revoke
any offer to present. =3D=3D About DebConf =3D=3D The annual DebConf conference is a technical and social forum for
Debian developers, sponsors, affiliates, and friends. It allows
various groups within Debian a chance to come together, network, and
share their work. Until we see you in the big city,
DebConf Talks Team [1] introduction written by Micah Anderson

Michael Banck: 30 Apr 2010

Science Track at Debconf10 This year's Debconf will probably have for the first time tracks pertaining to certain subjects. One of the proposed tracks will be "Math and Science and Debian" and I was asked to organize it. We have some talk proposals already, and we will have a panel discussion about Debian science packaging between the major contributors of the various packaging teams, but more talks are welome! Talks could be about science with/on Debian, or Debian development in scientific fields. Additionally, I would like to encourage contributions on three types of topics: Full length talks are not required, we could split up each topic among a number of people depending on interest and submissions. Talks can be officially submitted until the end of tomorrow (Saturday May 1st, 23h59 UTC), but even afterwards, if you plan to go to Debconf and would like to give a talk about Math, Science and Debian, please contact me and we will see what can be done. Especially contributions to the three topics above can be made later as the events will be registered already.

Erich Schubert: Oil spill: fortunately it was "just oil" - and not nuclear.

Face it: technology fails. Sometimes. Sooner or later.Statistics are a nice way to fool people, because they can't interpret the numbers properly. And usually this is done to make things sound less risky than they really are. But it's just too easy to lie with numbers. Say, they risk of a major incident at a plant is less than 1% (per year). Sounds good, doesn't it? But if you have 100 plants, that means it will happen about once a year. Make that 1000. Or even more.There are currently around 440 nuclear reactors in use for electricity. Probably some more the IAEA doesn't know about. There are definitely extra reactors in various military vehicles including unmaintained russian submarines. So make that 500. So looking forward to the next 200 years, we want to make sure things are not likely to happen at a rate of 0.001%. Unless our atomic plants are way more than 99.999% secure, we're likely to see a major nuclear accident sooner or later.Which brings us to a totally differnt scale when we look at photovoltaics or wind power. There are just so many more of these plants, that you can bet on some of them failing completely every day. The good news about these plants is that their damage potential is just so much lower. They might be as much as thousandfold more likely to fail than a nuclear plant. And there are probably a thousand more, too. But even if we completely underestimate the risk they just don't do as much damage even in the worst case. Much unlike a nuclear meltdown or such an oil spill. I bet noone would have thought of oil drilling being this risky. After all, it has been done for years.So how is your risk assessment of nuclear power? Is it less risky than oil drilling?

Asheesh Laroia: Mailing lists can wait

Most of my email comes to me via mailing lists. When someone asks a question or starts a discussion, I used to jump at the chance to be the first to reply with an answer. I don't want to unsubscribe; often, I know something that can move the conversation forward or help someone out. But I'm trying to allocate more time for myself away from computing these days. So now I let these discussions mature a little before I see them. All mailing list messages are delivered to a folder called OneDayDelayed. Every night, my server moves those messages to my INBOX. If someone else has already contributed what I would have, perfect! If not, I still have a chance to participate. Sometimes, if I have a long backlog of email, I disable that cron job, sometimes for up to two weeks. Being able to turn the cron job on and off really helps; when I have time to deal with mailing lists, I do. Otherwise, I know the messages will be there when I do have time. Since this OneDayDelayed change, I've gained have a sense of calmness and distance from them. I like that. (Note that if you add me to the CC: line of a mailing list thread, I will see your message immediately, for better or for worse.)

Russell Coker: Link Within

Good Things about LinkWithin For the last 10 weeks I ve been using the LinkWithin.com service to show links to other blog posts at the end of each post (the links are only shown to visitors of my blog not in the RSS feed, so people who read my posts through RSS syndication will miss this). The service shows excerpts of pictures from my blog at the bottom of each post to entice readers into reading other posts. When you click on a LinkWithin icon on my blog you visit a LinkWithin page that redirects you back to my page, so people who use that show up as new visitors being referred by LinkWithin. Currently this month LinkWithin is the fourth highest referrer to my blog, below Google, Reddit, and WebWombat.com.au so it is clearly doing some good in enticing people to read other posts that they might not otherwise read! Bad Things about LinkWithin The first problem with LinkWithin is that the Wordpress plugin was written by people who don t know much about Wordpress. Unlike every other plugin I use it doesn t support configuration options for the user in the database but instead has it hard coded in the PHP code! When you download a plugin from their web site it creates a custom zip that includes the PHP code automatically generated just for you! I have some friends using the same web server as me for running blogs, and I have to tell them you can install any plugin apart from LinkWithin it works for no-one but me . The next problem is that LinkWithin advertises that it will display related posts, it seems to be doing a poor job of that on my blog although admittedly only a minority of my blog posts have pictures so this limits what it has to work with. This has however inspired me to use more pictures in my posts. Another problem is that it produces web pages that are not valid XHTML, the following patch fixes this. --- /tmp/linkwithin.php 2010-02-22 02:09:47.000000000 +0000
+++ ./wp-content/plugins/linkwithin/linkwithin.php 2010-02-22 02:17:29.000000000 +0000
@@ -15,7 +15,7 @@
global $post, $wp_query, $linkwithin_code_start, $linkwithin_code_end; $permalink = get_permalink($post->ID);
- $content .= '<div class="linkwithin_hook" id="'.$permalink.'"></div>';
+ $content .= '<div class="linkwithin_hook" id="'.str_replace('/','-',$permalink).'"></div>';
$content = linkwithin_add_code($content);

return $content;
@@ -26,13 +26,13 @@
global $post, $wp_query, $linkwithin_code_start, $linkwithin_code_end; if ($wp_query->current_post + 1 == $wp_query->post_count)
- $embed_code = '<script>
+ $embed_code = '<script type="text/javascript">
<!-- //LinkWithinCodeStart
var linkwithin_site_id = 151382;
var linkwithin_div_class = "linkwithin_hook";
//LinkWithinCodeEnd -->
</script>
-<script src="http://www.linkwithin.com/widget.js"></script>
+<script src="http://www.linkwithin.com/widget.js" type="text/javascript"></script>
<a href="http://www.linkwithin.com/"><img src="http://www.linkwithin.com/pixel.png" alt="Related Posts with Thumbnails" style="border: 0" /></a>';
$content .= $embed_code;
Finally LinkWithin shows up badly on the FireBug analysis (see my previous post about using Firebug to speed up my blog) [1] see the below picture for details. As an aside, given that Google recommend Firebug it is rather ironic that Google Adsense related URLs cover the majority of the Firebug issues that are not caused by LinkWithin. demonstration of how Linkwithin lowers my page speed What Next? I ve sent email to the LinkWithin people about all these issues other than the FireBug problem reports, given that they haven t responded to some suggestions for over 10 weeks it seems hardly worth my effort in informing them of other issues. I m thinking of trying OutBrain.com again. 18 monthe ago I tried OutBrain but never got it working due to technical issues and then forgot about it. It has some similar features and may work better at least it has tech support people who respond to queries!

Matt Brubeck: Fennec on Android: user feedback and next steps

Last month I joined Mozilla as a UI engineer on the Fennec (Mobile Firefox) project. Firefox is already available for Nokia's Maemo platform, and now a group of Mozilla programmers are porting it to Google's Android OS. This Tuesday they made an early preview build available for public feedback. Until now, the only people working on Firefox for Android were platform developers getting the back-end code to build and run. This week was the first time most other people including me got to try it out. We front-end developers and designers are now starting to adapt the user interface to Android. (The preview build used the look and feel of Firefox for Maemo, designed for rather different hardware and software.) Because we are an open source project, we like to share our work and hear your feedback even at this early stage of development. While I wasn't directly involved in the Android development effort, I spent some of my spare time this week talking to users via Twitter and our Android feedback group. Here's what I heard, in rough order of importance to users, plus some information on our future plans.1 We don't have a regular schedule yet for releasing new builds on Android. Once we get the code merged and automated build servers configured, we'll publish nightly builds of Firefox for Android alongside our Maemo and desktop nightlies. Later this year we will have alpha and beta versions, and hopefully a stable release. Until then, you can follow @MozMobile or Vlad (@vvuk) to hear about any new previews.
  1. Please remember I am still new to the project, and cannot speak for the whole team. This is a personal blog, not a Firefox roadmap!

Russell Coker: CPL Still Sucks

I previously described my experience with Computers and Parts Land (CPL) [1] who gave me a product that didn t do what I wanted (because they thought that they knew better than me) and then gave me attitude when I returned it. As it s almost a year since that incident I had to buy something else from them to use the credit note as I decided that it s not worth making a Trade Practices Act (TPA) [2] issue out of it. Thanks Tim for the suggestion though. Firstly I went to their new store that is advertised on their web site. The people at the new store refused to honor the credit note (they probably hoped that I would just let it go and give them an extra $60 in profit). Claiming that I need to take a credit note back to the store it came from is bogus. Anyway I went back to the original store and bought a new 1.5TB SATA hard drive which seems to work well enough. The service was really slow, I was the only customer in the store and there were several employees not doing much but it still took them ages to print a receipt and give me the hard drive. It will be the last CPL purchase I ever make. In the unlikely situation that there is ever any new gear that is only sold by CPL then I will wait a year and buy it on ebay. It would be good if someone started working as a TPA complainant. They would take donations from dis-satisfied customers who want to pay for an investigation of a shonky company (I d pay $10 for an investigation of CPL). When enough donations were collected they would buy some stuff and make a video of every interaction with the employees and then launch a private legal action under the TPA if they do anything wrong. The TPA complainant would get to keep everything that they buy, any excess donation money after proving the case, and any money offered by the company to settle law suits. While it s not a good use of my time and money to go after CPL directly it would be good to give some money to someone who would then see them dealt with properly.

29 April 2010

Sylvestre Ledru: Google Summer of code at Scilab

First, a quick news. Scilab 5.2.2 has been released a few days and he now available in the Debian archive (Debian Unstable for now).
It is mostly a performance improvements and bug fixes release. See the changelog. Second, for a second year, Scilab is part of the Google Summer of Code v2010. I am admin (and also mentor) this year again.
Google just released the list of accepted students. We have 9 students (against 7 last year). 2 are working on experimental projects, the other should have a finished result by the end of the GSoC. Extends Scilab UI Control - Han Dong
Extension of Scilab User Interface construction (uicontrol) by providing more elements (tree, image display, etc). Simulink import in Xcos - Jerzy Zagorski
Importation of most of the Simulink schemas from Xcos. Cumulative distribution function improvements - Michael Zhang
Improvement and extension of the current cumulative distribution functions (CFD) into Scilab. Metanet and Boost.graph - Bal a Rai evi
Extension of the Metanet capabilities in term of graphs by Boost.graph SOAP Client/server - Artem Glebov
Making available from Scilab both SOAP client and server capabilities as an ATOMS module. Database module + fuzzySQL - Igor Gridchyn
Allow to access to most databases systems from Scilab. Based on this work, the FuzzySQL will be introduced, a SQL extension to allow flexible conditions in queries. Python import - Baozeng Ding
Introduction of a mechanism to load and use Python code (objects in particular) from Scilab. Experimental projects:
Use Eigen into Scilab - Joseph Fahnbulleh
Base some components of the Scilab core code on the Eigen library which is a state-of-the-art C++ linear algebra library. This work will be a joint mentoring between Ga l Guennebaud from the Eigen team and Scilab R & D team. Hybrid Automata module - Ievgen Ivanov
Provide a convenient environment for direct modeling of hybrid automata in Scilab/Xcos.

David Welton: Erlang vs node.js

I've written about Erlang in the past, and my suspicion that, soon or later, other languages/systems will come along and "eat its lunch". Scala is one such potential contender. Another that has been gaining some visiblity lately is node.js, a simple framework for creating networked applications on top of Google's V8 Javascript engine. I should define what Erlang's lunch is a little bit better before we go on. Erlang does several things really, really well; better than many existing, mainstream systems:
  1. Concurrency - the Actor model that it uses is much easier and more pleasant than dealing with threads.
  2. Distributed systems - Erlang makes this fairly easy and pleasant as well.
  3. Fault tolerant systems - using its distributed powers, Erlang is good for writing things like telephone switches that can't spend any significant time down.
Of these, I think the big one is concurrency. Distributed systems are nice, but not a critical point for most people. Same goes with fault-tollerant: those who need it really need it, but the rest of us are willing to make some compromises. Our web applications are important, but generally not something where people's lives hang in the balance. How does Erlang do "concurrency"? The developer creates Erlang "processes" that interact with one another by passing messages. These are not real, OS level processes, though, and this is critical to how Erlang operates. Since these processes must all coexist within one real, system level process, it is absolutely essential that no operation that they perform hangs the entire system! The Erlang runtime system is built around this concept. Any Erlang process can do things like read and write to the disk or network, or have a "while (true) .... " loop (it doesn't actually look quite like that in Erlang, but that's the idea), and it won't wedge the system. This knowledge is also critical when you want to interface Erlang to the outside world: if your C library contains a routine that might block for a long time, you can't just call it from Erlang as it won't be a well-behaved part of Erlang's world (there are ways around this of course, but make life a bit more complicated for the developer). All this is done with Erlang's scheduler: each Erlang process gets a number of operations it can run before some other process gets to run, so even our while loop will only run for a bit before the system moves on to something else. IO is rigorously done with non-blocking calls internally in order to keep that from becoming an issue. No other system that I know of has such a focus on being non-blocking, and node.js is no exception: a while(true) loop is perfectly capable of wedging the system. Node.js works by passing functions (closures, in many cases) around so that work can be performed as needs be. However, the actual functions that run actually do block the system, and thus must be written in order to not run for too long. Also, and this is important, Node.js also does its best to make IO non-blocking by default, so that you don't have to worry about IO calls. Node.js isn't up to the level Erlang is at, because it requires more manual intervention and thinking about how to do things, but it's probably "good enough" for many tasks. How often do you really write code with so many calculations that it slows things down? Not often in my case - the real world problem I most often encounter is IO, and node.js does its best to make that non-blocking, so that it can be handled in the "background" or a bit at a time, without wedging the system. And if you really needed to write a long-running calculation (say you want to stream the digits of PI or something), you can break up your calculation manually, which may not be quite as elegant as Erlang, but is "good enough" for many people. "Good enough" concurrency, combined with a language that is at least an order of magnitude more popular than Erlang, and a fast runtime, combined with ease of use in general (it's way easier to get started with node.js than with most Erlang web stuff) make for a system that's likely to do fairly well in terms of diffusion and popularity, and is going to "eat some of Erlang's lunch". Or perhaps, rather than actually taking users away from Erlang, it's likely to attract people that might have otherwise gone to Erlang.

28 April 2010

John Goerzen: Download A Piece of Internet History

Back in the early 1990s, before there was a World Wide Web, there was the Internet Gopher. It was a distributed information system in the same sense as the web, but didn t use hypertext and was text-based. Gopher was popular back then, as it made it easy to hop from one server to the next in a way that FTP didn t. Gopher has hung on over the years, and is still clinging to life in a way. Back in 2007, I was disturbed at the number of old famous Gopher servers that had disappeared off the Internet without a trace. Some of these used to be known by most users of the Internet in the early 90s. To my knowledge, no archive of this data existed. Nobody like archive.org had ever attempted to save Gopherspace. So I decided I would. I wrote Gopherbot, a spidering archiver for Gopherspace. I ran it in June 2007, and saved off all the documents and sites it could find. That saved 40GB of data, or about 780,000 documents. Since that time, more servers have died. To my knowledge, this is the only comprehensive archive there is of what Gopherspace was like. (Another person is working on a new 2010 archive run, which I m guessing will find some new documents but turn up fewer overall than 2007 did.) When this was done, I compressed the archive with tar and bzip2 and split it out to 4 DVDs and mailed copies to a few people in the Gopher community. Recently, we ve noted that hard disk failures have hobbled a few actually maintained Gopher sites, so I read this archive back in and posted it on BitTorrent. If you d like to own a piece of Internet history, download the torrent file and go to town (and please stick around to seed if you can). This is 15GB compressed, and also includes a rare video interview with two of the founders of Gopher. There are some plans to potentially host this archive publicly in the manner of archive.org; we ll have to wait and see if anything comes of it. Finally, I have tried to find a place willing to be a permanent host of this data, and to date have struck out. If anybody knows of such a place, please get in touch. I regret that so many Gopher sites disappeared before 2007, but life is what it is, and this is the best snapshot of the old Gopherspace that I m aware of and would like to make sure that this piece of history is preserved.

Sam Hartman: Debconf 10 Enterprise Track

I ve been asked by the Debconf 10 talks team to coordinate a Debian Enterprise track at the upcoming Debconf 10 conference. I m really excited about this, and I could use your help. From my standpoint, this all started with a BOF proposal looking at better coordination and what is missing in Debian enterprise integration. The track will also include talks and other events on what exciting things are happening with Debian in the enterprise. I need your help in the form of suggestions for talks, panels and the like (especially if you would be willing to give the talk or coordinate an activity). We re under a fairly tight deadline; ideally proposals would be in by May 1, but if they are not, it s still definitely worth discussing with me. From my standpoint, topics include: Obviously, many other things could fit in and I d be interested in your ideas as well.

Andrew Pollock: [life] 17 days

Getting close now. Sarah had another ultrasound yesterday, just to check things. Unfortunately I couldn't be there due to a self-inflicted scheduling screw up. Apparently you could see hair on the ultrasound, so I'm a bit bummed to have missed it. The baby has "dropped", so that's good. Her head's still down, and she's swallowing, so that's all good too. Weight-wise she's now in the 38th percentile. We're booked in for a C-section on the 14th, unless she comes of her own free will between now and then (which is what we'd prefer), in which case our OB will allow an attempt at a VBAC. Our OB is reluctant to allow the pregnancy to go past 39 weeks, given prior history and Sarah's heart, and they don't induce for VBACs, so it's going to require some planetary alignment to avoid a C-section. The whole heart thing still being an issue was only a fairly recent discovery for us. Sarah's cardiac surgeon had told us we could go forth and have as many kids as we wanted, but it turns out the fine print in that statement is that while the previous aneurysm has been repaired, she has an undiagnosed connective tissue disorder, so there's no telling what might happen with the rest of her aorta in the future. So her OB is just doing what a good high-risk OB should do, and being extra paranoid, on two fronts. So we've got a couple of weeks to try and convince our daughter to come more than a week and 2 days early. I don't think we have statistics on our side, unfortunately (although I had a surprisingly hard time finding concrete statistics on delivery dates relative to due dates for first-time deliveries versus subsequent deliveries).

27 April 2010

Matthew Garrett: Radeon reclocking

Alex released another set of Radeon power management patches over the weekend, and I've been adding my own code on top of that (Alex's patches go on top of drm-next, mine go on top of there). I've left it stress-testing for a couple of hours without it falling over, which tells me that it's stable enough that I can feel smug. This is a pleasing counterpoint to the previous experiences I've been having, which have been rife with a heady mixture of chip lockups or kernel deadlocks. It turns out that gpus are hard.

There's a few things you need to know about gpus. The first is that if they're discrete devices they typically have their own memory controller and video memory. The second is that there's an impressive number of ways that you can end up touching that memory. The third is that they tend to get upset if something tries to touch that memory and the memory controller is in the middle of reclocking at the time.

The first and most obvious use of video memory is by the gpu itself. Accelerated operations on radeon are carried out by sending a command packet to the command processor. This is achieved by sharing a ring buffer between the cpu and the gpu, with the gpu reading packets out of that ring buffer and performing the operations contained within them. Many of these operations will touch video memory (that being the nature of most things you want a gpu to do), and if that happens bad things occur. Like the card locking up and potentially taking your PCI bus with it.

So, obviously, we don't want that to happen. The first thing we do is take a mutex that blocks any further accelerated operations from being submitted by userspace. Then we wait until we get an interrupt from the gpu telling us that the display engine has gone idle. The problem here is that we don't have a terribly good idea of how many more operations there are to complete and we don't know how long each of those operations is going to take, but this is less bad than some of the alternatives[1]. Jerome Glisse has some ideas on how to improve this to require less waiting, but the effects should still be pretty much invisible to the average user.

So we've stopped the command processor touching ram. Everything's good, right?

Well, not really. The obvious problem is that users typically want to display something, so there's a separate chunk of chip that's repeatedly copying video memory over to your monitor. That's got to go too. Thankfully, there's a convenient bit in the crtc registers that lets us turn that off, but the pretty unsurprising downside is that your screen goes blank while that's happening. So we don't want to do that. Instead, we try to perform the reclock while there's nothing being displayed on the screen - that is, while we're in the screen region where a crt's electron gun would be scanning back from the bottom of the screen to the top. It turns out that rather a lot of display assumptions depend on this happening even if there's no crt, no electron gun and no thick sheet of glass with a decent approximation of vacuum behind it, so we get to do this even if we're displaying to an LVDS. And we have about 400-500 microseconds to do it - an almost generous amount of time.

So we ask the hardware to generate an interrupt when we enter vblank and then we reclock. Except the hardware has an irritating habit of lying - sometimes we get the interrupt a line or two before vblank, sometimes we get it after we've already gone out the other side. Vexing, and not entirely solved yet - so sometimes you'll still get a single blank frame during reclock. But there are plans, and they'll probably even work.

At this point the acceleration hardware isn't touching the memory and the scanout hardware isn't touching the memory. Except it still crashes under some workloads. This one took me longer to track down, but the answer turned out to be pretty straightforward. Not all operations are accelerated. When they're not accelerated they have to be done in software. That means that the CPU has to write to the video memory itself. I'm sure you can see where this is going. This was fixed without too much trouble once I'd finished picking through the driver to work out every location where objects might be mapped into the CPU's address space, at which point it's a simple matter of unmapping them and blocking the fault handler from remapping them until the reclock is finished. Linux, thankfully, has lots of synchronisation primitives. And now everything works.

Except when it doesn't. This took a final period of head scratching, followed by the discovery that ttm (the memory allocator used by radeon) has a background thread that would occasionally fire to clean up objects. And by clean up objects, I mean change their mapping - which means updating their status in the gart, which means touching video memory. So, let's block that. And that tripped me off to the fact that even if it couldn't submit new commands, the CPU could still create or destroy objects - with the same consequences.

So, once all of these are blocked, video memory is quiescent and we can do what we want. And we do, at least once I'd sorted out the bits where I was taking locks in the wrong order and deadlocking. Depending on the powerplay tables available on your card we'll chose different rates and so your power savings will vary heavily depending on the values that your vendor provided, but the card I'm testing on sees a handy 30W drop at idle. Right now we're only changing clocks and not dropping voltage so there's potentially a little more to come.

While getting this stable was pretty miserable, the documented entry points for clock changing made a lot of this easier than it would otherwise have been. It's also probably worth noting that Intel's clock configuration registers are entirely missing from any public documentation and the dirver Intel submitted to make them work in their latest chips appeared to have been deliberately obfuscated, so thanks to AMD for making all of this possible.

[1] It's possible to insert various command packets that either indicate when they've passed or stall until a register value gets updated, but these either cause awkward problems with the cp locking or mean that the gui idle functionality never goes idle, so they're not ideal either.

Michal &#268;iha&#345;: Let the fun begin

We've yesterday announced students accepted this year to Google Summer of Code. The number of our projects is increasing every year and the number of applications grows even more. So it was hard to choose this year knowing that we can mentor only limited amount of students (we would happily do two or three more projects, but we simply lack manpower to mentor them). So the students are chosen and I hope they will produce good results and all phpMyAdmin users will benefit from their contributions. If you want to follow the projects more closely, Planet phpMyAdmin is the right place, first student blog posts are already appearing there.

Phil Hands: The rise of the Clan MacDebian

I've asked the weavers to start doing their job today, so with luck the tailors will still have time to make kilts ready to wear for DebConf10.If you've been wondering if you'd like some Debian Tartan, now's your chance. We've ordered a little over 16 yards more than we need, which will be enough to make about 2 extra kilts and a few ties (depending on exact details), so order early to avoid disappointment.Of course, kilts are not the only thing one can make out of tartan. The tailors make other things, but there's nothing stopping you from buying some cloth by the yard and being inventive (it costs about 25.00 GBP per yard). We'll also be making ties, so if you're interested in one of those, please add yourself to the relevant table on the wiki, and I'll get in touch with prices etc.Full details of this silliness are available on the wiki page -- send any questions to phil at hands dot comUpdate 16:00 UTC 27th: one 8-yard kilt snapped up already (numbers above updated)

Gunnar Wolf: Probability, mindlessness and periodicity

I bought my current cell phone in mid January 2008, which means I have had it for ~830 days. I tend not to lose sight of the few things I keep on me at all times - Keys, wallet and cell phone. In this 830 day period, I doubt I have forgotten my phone for more than three days. Ok, lets just add some extra five days, for times I have travelled without the phone charger, resulting in me not having a phone available. So, eight days out of 830 gives us that the probability for me accidentally not having access to my phone for a whole day is close to 0.01, or 1% P(stupid) 0.01. I am not a heavy phone user. Far from it, I tend not to like using the phone. My calls are always kept to the minimum necessary, and I would not be surprised if I used my phone typically less than twice per week. However, there is a recurring event that makes my phone ring more often during certain very well defined periods of the year, with 1/365 probability. So, lets call this P(birthday) 0.00273. Yesterday I spent the day with some good, very long time friends. We did the April Ciclot n - I biked for 42 very nice kilometers, they did 36 (as we met 8Km away from my home ;-) )... but yes, after a while, I left my phone at my friends' table. So I guess today the phone will ring way more insistently than usual. At a table far away from home. With a probability of P(stupid birthday) 2.73 10 . So I'm facing the enjoyment of a very improbable day!

Dirk Eddelbuettel: R Project and Google Summer of Code: Welcome to our students!

A few hours ago, I sent the following to both the R development list and the informal R / GSoC list:
Date: Mon, 26 Apr 2010 15:27:29 -0500
To: R Development List <r-devel>
CC: gsoc-r <gsoc-r>
Subject: R and the Google Summer of Code 2010 -- Please welcome our new students!
From: Dirk Eddelbuettel <edd>
Earlier today Google finalised student / mentor pairings and allocations for
the Google Summer of Code 2010 (GSoC 2010).  The R Project is happy to
announce that the following students have been accepted:
   Colin Rundel, "rgeos - an R wrapper for GEOS", mentored by Roger Bivand of
      the Norges Handelshoyskole, Norway
   Ian Fellows, "A GUI for Graphics using ggplot2 and Deducer", mentored by
      Hadley Wickham of Rice University, USA
   Chidambaram Annamalai, "rdx - Automatic Differentiation in R", mentored by
      John Nash of University of Ottawa, Canada
   Yasuhisa Yoshida, "NoSQL interface for R", mentored by Dirk Eddelbuettel,
      Chicago, USA
   Felix Schoenbrodt, "Social Relations Analyses in R", mentored by Stefan
      Schmukle, Universitaet Muenster, Germany
   Details about all proposals are on the R Wiki page for the GSoC 2010 at
   http://rwiki.sciviews.org/doku.php?id=developers:projects:gsoc2010
The R Project is honoured to have received its highest number of student
allocations yet, and looks forward to an exciting Summer of Code.  Please
join me in welcoming our new students.
At this time, I would also like to thank all the other students who have
applied for working with R in this Summer of Code. With a limited number of
available slots, not all proposals can be accepted -- but I hope that those
not lucky enough to have been granted a slot will continue to work with R and
towards making contributions within the R world. 
I would also like to express my thanks to all other mentors who provided for
a record number of proposals.  Without mentors and their project ideas we
would not have a Summer of Code -- so hopefully we will see you again next
year. 
Regards,  
Dirk (acting as R/GSoC 2010 admin)

C.J. Adams-Collier: GSoC 2010

Earlier this month, I applied to the Mono Project (and the University of Washington, and Ubuntu, and Debian, and The Perl Foundation) requesting a mentor to get Perl6 hosted on the DLR. Last Tuesday, Miguel contacted me and asked that I chat with Michael Hutchinson about possibly taking up a different project. It seems that the group did not have any mentors who felt comfortable mentoring the Perl6 project. After a bit of consideration, I agreed to modify my application and take up a project to revive the regular expression compiler from 2.2. Today, the project was officially accepted, and I met with my mentor for the first time (hi Rodrigo!). I will also be working with Matthew Wilson (aka @diakopter), since he has purportedly implemented a number of regex-to-IL compilers ;) He also offered to mentor me if The Perl Foundation had accepted my application, and since he has already implemented a perl6 compiler in javascript, I have been looking forward to poking some code with him. Although the GSoC doesn t officially get started until 5/24, I m making a git-svn checkout now. I ve always committed the code directly to svn, but I ve enjoyed working with git, and it seems about time to start contributing via git-svn. It will be easier to have local branches this way, too. Anyway, I m looking forward to it ;)

26 April 2010

Jan Hauke Rahm: Talking about Free Software

When I first started using Free Software yes, that s quite some time ago I found it the right thing to do. It felt like I was first time ever really controlling what my computer does. Took just a bit of time to notice I really never knew anything about it. Well, almost at least. Then when I started contributing to Free Software this obviously changed. One of the freedoms is to learn from the code you re running. And I did it by reading, contributing, doing nonsense with it, reimplementing it you know the deal. Starting to work on Debian was something that just evolved out of it. Doing so gave me the opportunity to give back. I never achieved that actually, though. At least I don t consider my contributions even remotely sufficient to make up for the great deal of software that I use sometimes without noticing it. Still there s lots of stuff that I don t have a clue of and I rely on other developers to do their (volunteer) job. Otherwise my knowledge wouldn t help much. I could never continue kernel development if those kernel hackers all stepped back for instance. I guess, it s getting more and more difficult the closer the piece of code is to hardware. Another thought I had was about Free Software in general. It s more than just the code (although it might be the most important part), it s also a concept that needs to be communicated. Firefox on Windows shows how Free Software can have a big market share in an otherwise closed source environment. Yet I m not sure all users know about it being open source and what that means (hear them talk about Freeware instead). And who is dealing with this conceptual stuff? I decided to do something about all that. Now I don t really have money but a bit I thought I could share. And I could set yet another sign by publicly talking about it, or casually mentioning that I consider Free Software and Linux a good thing. So you will see a mail signature now below all my mails. I always wondered what one might gain with such and now I found a reason to include one. It reads:
Debian Developer                                 www.debian.org
Member of the Linux Foundation                    www.linux.com
Fellow of the Free Software Foundation Europe      www.fsfe.org

Next.

Previous.