Search Results: "ag"

7 November 2015

Guido G nther: Debian Fun in October 2015

Debian LTS October was the sixth month I contributed to Debian LTS under the Freexian umbrella. In total I spent four hours working on: Besides that I did CVE triaging of 16 CVEs to check if and how they affect oldoldstable security as part of my LTS front desk work. I also added some very basic indentation support to our CVE/list Emacs major-mode on non LTS time. Other Debian stuff

Wouter Verhelst: Minidebconf Cambridge

My main reason for being here was to join the Debconf video team sprint. But hey, since I'm here anyway, why not join all of it, right? Right. Apart from the video team stuff that I've been involved in, I've been spending the last few days improving NBD: There're a few more things that I'd like to see fixed, but after that I'll probably release 3.12 (upstream and in Debian) sometime later this week. With that, I'll be ready to start tackling Debian bug #796633, to provide proper systemd support in nbd-client.

Mart n Ferrari: Tales from the SRE trenches: Coda

This is the fifth -and last- part in a series of articles about SRE, based on the talk I gave in the Romanian Association for Better Software. Previous articles can be found here: part 1, part 2, part 3, and part 4.

Other lessons learned Apart from these guidelines for SRE I have described in previous posts, there were a few other practices I learned at google that I believe could benefit most operations teams out there (and developers too!).

Automate everything It's worth repeating, it is not sustainable to do things manually. When your manual work grows linearly with your usage, either your service is not growing enough, or you will need to hire more people than there are available in the market.

Involve Ops in systems design Nobody knows better than Ops the challenges of a real production environment, getting early feedback from your operations team can save you a lot of trouble down the road. They might give you insights about scalability, capacity planning, redundancy, and the very real limitations of hardware.

Give Ops time and space to innovate If there is so much emphasis on limiting the amount of operational load put on SRE, that freed time is not only used for creating monitoring dashboards or writing documentation. As I said in the beginning, SRE is an engineering team. And when engineers are not running around putting fires off all day long, they get creative. Many interesting innovations come from SREs even when there is no requirement for them.

Write design documents Design documents are not only useful for writing software. Fleshing out the details of any system on paper is a very useful way to find problems early in the process, communicate with your team exactly what are you building, and convince them why it is a great idea.

Review your code, version your changes This is something that is ubiquitous at Google, and not exclusive to SRE: everything is committed into source control: big systems, small scripts, configuration files. It might not seem worth it at the beginning, but having complete history of all your production environment is invaluable. And with source control, another rule comes into play: no change is committed without first being reviewed by a peer. It can be frustrating having to find a reviewer and wait for the approval for every small change, and some times reviews will be suboptimal, but once you get used to it, the benefits greatly out-weights any annoyances.

Before touching, measure Monitoring is not only for sending alerts, it is an essential tool of SRE. It allows you to gauge your availability, evaluate trends, forecast growth, perform forensic analysis. Also very importantly, it should give you the data needed to take decisions based on reality and not on guesses. Before optimising, find really where the bottleneck is; do not decide which database instance to use until you saw the utilisation of the past few months, etc. And speaking of that... We really need to have a chat about your current monitoring system. But that's for another day.
I hope I did not bore you too much with these walls of text! Some people told me directly they were enjoying these posts, so soon I will be writing more about related topics. In particular I would like to write about modern monitoring and Prometheus. I would love to hear your comments, questions, and criticisms. Comment

Neil Williams: Vmdebootstrap sprint

At the miniDebConfUK in Cambridge, November 2015, there was a vmdebootstrap sprint. vmdebootstrap is written in python and the sprint built on the changes I made during DebConf15. The primary aim was to split out the code from a single python script and create modules which would be useful to other tools and make the source code itself easier to follow. Whilst doing this, I worked with Steve McIntyre to implement UEFI support into vmdebootstrap. This version reached experimental shortly after DebConf15. At the sprint, the squashfs support in vmdebootstrap was improved to be more useful in the preparation of live images rather than simply using mksquashfs as a compression algorithm for the entire image. I also improved and extended the documentation for vmdebootstrap. vmdebootstrap is now extensible and modular. Iain Learmonth used this new modular support in the development of live-build-ng which uses vmdebootstrap, live-boot and live-config to replace the role of live-build in the generation of official Debian live images by the debian-cd team. Iain Learmonth demonstrated that the new support in vmdebootstrap can be used to create working live images, including adding support for live images on any architecture supporting the Linux kernel in Debian and adding support for not only Grub but UEFI as well. A UEFI grub live image built by live-build-ng was demonstrated after only two days work on the vmdebootstrap base, wrapping support from live-boot and live-config with customisation for UEFI and grub config. vmdebootstrap and live-build-ng have been explicitly designed within the debian-cd team to remove the need to run live-build to create official Debian live images, replacing live-build with live-build-ng and the vmdebootstrap support. This brings working support for multiple architectures and UEFI to live images. To support the new functionality, the vmdebootstrap and debian-cd team have long sought a test suite for vmdebootstrap and Lars Wirzenius implemented one during the vmdebootstrap sprint. We now have fast tests with a pre-commit hook and build tests which are best with a local mirror. The test suite uses cmdtest & yarn. This test suite will be used for all future changes patches will not be accepted if the test suite fails and any substantially new code must provide working test cases. The fast tests can run without sudo, I expect to be able to sort out ci.debian.net testing for the fast tests too. There is also an outline for testing vmdebootstrap builds in (localhost) LAVA using a lava-submit.py example. All this work will arrive in unstable as vmdebootstrap 1.2 soon and is now in the master branch. The old codehelp/modules branch has been merged and removed.

Andrew Cater: Mini Debconf ARM Cambridge 1115 7 November

Between talks - coding and testing going on in the row infront of me - vmdebootstrap rework for Debian Live CD still going on.

Beaglebone SBC hanging from a Thinkpad in the row behind

Others talking - most getting coffee

Next up - Daniel Silverstone on Lua

6 November 2015

Andrew Cater: MiniDebconf Cambridge ARM, Cambridge 1020 6 November

Back here. Now a silent room - seven people on laptops in the Video Sprint rooom. A couple of low-key mutterings as various "stuff" is attempted to be ported.

Last night most folk watched the fireworks - November 5th is the celebration in UK which involves fireworks. [Obligatory Wikipedia link https://en.wikipedia.org/wiki/Guy_Fawkes_Night ]

Cambridge has a huge free public display and it was apparently well worth watching.

A side conversation on keyboards - Belgians, UK English, Germans - but most of us have US English layout in our touch typing fingers, I suspect. Changing keyboards is fun :)


I may have been talked into packaging something for Debian for the first time in a very long time :)







Ian Campbell: Cambridge mini-Debconf

I am currently attending the mini-Debconf being held in space generously provided by ARM's offices in Cambridge, UK. Thanks to ARM and the other sponsors for making this possible. Yesterday I made a pass through the bug list for the Xen packages. According to the replies I have received from the BTS I looked at and acted on: Phew! Today I expect more of the same, starting with seeing where the new information in #799122 takes me.

Matthew Garrett: Why improving kernel security is important

The Washington Post published an article today which describes the ongoing tension between the security community and Linux kernel developers. This has been roundly denounced as FUD, with Rob Graham going so far as to claim that nobody ever attacks the kernel.

Unfortunately he's entirely and demonstrably wrong, it's not FUD and the state of security in the kernel is currently far short of where it should be.

An example. Recent versions of Android use SELinux to confine applications. Even if you have full control over an application running on Android, the SELinux rules make it very difficult to do anything especially user-hostile. Hacking Team, the GPL-violating Italian company who sells surveillance software to human rights abusers, found that this impeded their ability to drop their spyware onto targets' devices. So they took advantage of the fact that many Android devices shipped a kernel with a flawed copy_from_user() implementation that allowed them to copy arbitrary userspace data over arbitrary kernel code, thus allowing them to disable SELinux.

If we could trust userspace applications, we wouldn't need SELinux. But we assume that userspace code may be buggy, misconfigured or actively hostile, and we use technologies such as SELinux or AppArmor to restrict its behaviour. There's simply too much userspace code for us to guarantee that it's all correct, so we do our best to prevent it from doing harm anyway.

This is significantly less true in the kernel. The model up until now has largely been "Fix security bugs as we find them", an approach that fails on two levels:

1) Once we find them and fix them, there's still a window between the fixed version being available and it actually being deployed
2) The forces of good may not be the first ones to find them

This reactive approach is fine for a world where it's possible to push out software updates without having to perform extensive testing first, a world where the only people hunting for interesting kernel vulnerabilities are nice people. This isn't that world, and this approach isn't fine.

Just as features like SELinux allow us to reduce the harm that can occur if a new userspace vulnerability is found, we can add features to the kernel that make it more difficult (or impossible) for attackers to turn a kernel bug into an exploitable vulnerability. The number of people using Linux systems is increasing every day, and many of these users depend on the security of these systems in critical ways. It's vital that we do what we can to avoid their trust being misplaced.

Many useful mitigation features already exist in the Grsecurity patchset, but a combination of technical disagreements around certain features, personality conflicts and an apparent lack of enthusiasm on the side of upstream kernel developers has resulted in almost none of it landing in the kernels that most people use. Kees Cook has proposed a new project to start making a more concerted effort to migrate components of Grsecurity to upstream. If you rely on the kernel being a secure component, either because you ship a product based on it or because you use it yourself, you should probably be doing what you can to support this.

Microsoft received entirely justifiable criticism for the terrible state of security on their platform. They responded by introducing cutting-edge security features across the OS, including the kernel. Accusing anyone who says we need to do the same of spreading FUD is risking free software being sidelined in favour of proprietary software providing more real-world security. That doesn't seem like a good outcome.

comment count unavailable comments

Norbert Preining: Japanese TeX User Meeting Support for Japanese in TeX (Live) present state and future?

Tomorrow the Japanese TeX User Meeting 2015 will be held in Tokyo. Since I have come to Japan I have attended more or less every year this meeting, and in 2013 we could get the (international) TeX User Group Meeting to Tokyo and had a joint conference TUG 2013, which was a great success. (update 2015-11-09: slides added at the end) Japanese-TeX
During tomorrows meeting I will give a talk on Support for Japanese in TeX (Live) present state and future?:
The support for typesetting Japanese in TeX, in particular TeX Live, has been improved over the last few years considerably, so that at the current point we can ask ourselves what is next? This talk reviews a bit of the history of Japanese typesetting support in TeX Live, and discusses future development directions.
For those interested, here is a bit of an extended abstract. With a set of Japan specific TeX engines, the Japanese TeX Users always had a hard time setting up a proper TeX environment. Besides compiling the programs for each new version again, users had to install packages from a variety of sources, which often involved complicated routines. Fortunately, due to the excellent support on the TeX Q&A these difficulties were manageable. Compiling the Japanese TeX engines was first part of the pteTeX project, which adjusted teTeX to include also the required programs. Compilation was done by obtaining the source code of teTeX, plus patches that implement the Japanese TeX engines and support in driver programs. After the end of development of teTeX, TeX Live became the main TeX distribution system, and ptexlive was born to adjust the TeX Live sources to include the Japanese TeX engines. But fundamentally the same procedure remain: Get the source code, patch in additional functionality, compile and install. The last five years have seen a step-wise inclusion of various Japan-related programs in the main TeX Live source repository. Starting with pTeX (and pbibtex etc) and adjustments to dvipdfmx, followed by upTeX, and later on also (u)pmetapost, the main TeX Live source repository now contains all (with the exception of pxdvi) programs related to the Japanese TeX environment. At the same time, developers of various packages started to upload their packages to CTAN, which automatically made these packages available in TeX Live. Thus, the current status is quite satisfactory, since a normal TeX Live installation, without the need for any compilation, provides all necessary programs and packages. What remains a problem are inter-release updates of the binaries, when bugs in various engines are found. Future developments Besides the actual development of packages, and in particular luatexja, I see possible improvements in the following area: Summing up, I am quite happy to see that both the Japanese community, as well as the Korean community, have now taken the step to make their achievements available in an easy and straightforward way for everyone using TeX Live, and I am sure that the future will bring further improvements!
The slides accompanying the talk can be downloaded here: texconf15-texlive-japan.pdf. I also talked briefly on CJK font integration for Ghostscript, these slides can be downloaded here: texconf15-cjk-gs.pdf.

5 November 2015

Steve Kemp: lumail2 approaches readiness

So the work on lumail2 is going well, and already I can see that it is a good idea. The main reason for (re)writing it is to unify a lot of the previous ad-hoc primitives (i.e. lua functions) and to try and push as much of the code into Lua, and out of C++, as possible. This work is already paying off with the introduction of new display-modes and simpler implementation. View modes are an important part of lumail, because it is a modal mail-client. You're always in one mode: This is nothing new, but there are two new modes: Each of these modes draws lines of text on the screen, and those lines consist of things that Lua generated. So there is a direct mapping:
ModeLua Function
maildirfunction maildir_view()
indexfunction index_view()
messagefunction message_view()
luafunction lua_view()
With that in mind it is possible to write a function to scroll to the next line containing a pattern like so:
function find()
   local pattern = Screen:get_line( "Search for:" )
   -- Get the global mode.
   local mode = Config:get("global.mode")
   -- Use that to get the lines we're currently displaying
   loadstring( "out = " .. mode .. "_view()" )()
   -- At this point "out" is a table containing lines that
   -- the current mode wishes to display.
    -- .. do searching here.
end
Thus the whole thing is dynamic and mode-agnostic. The other big change is pushing things to lua. So to reply to an email, populating the new message, appending your ~/.signature, is handled by Lua. As is forwarding a message, or composing a new mail. The downside is that the configuration-file is now almost 1000 lines long, thanks to the many little function definitions, and key-binding setup. At this rate the first test-release will be out at the weekend, but API documentation, and sample configuration file might make interesting reading until then.

Steinar H. Gunderson: Let's Encrypt Varnish plugin

I made a Varnish authentication plugin for the Let's Encrypt client. I dislike the huge amount of magic and layers of, well, stuff in the client, but the project is hugely important for the web, and I doubt there will be another ACME client anytime soon, so I can just as well get on the bandwagon. :-) It's really ugly. But it works for me.

Gunnar Wolf: WTF @omarfayad Should *all* computer activity be illegal now? #LeyFayad

Last week, Senator Omar Fayad presented one of the prime examples of a poorly redacted law that, if enacted, will make basically any way of computer use illegal. And yes, even if he states this is merely a draft, it has so many factual and conceptual errors that there is no way to trust sanity can be regained at any point. Oh, and before I continue with this rant: If the topic interests you, I suggest you to read the 10 key points about Ley Fayad, the worst Internet initiative in history, published by r3d.mx. [update] An English equivalent of the work at r3d, at revolution-news.com: #LeyFayad: The Worst Bill in Internet History The full text (in Spanish, of course) for the law initiative is available at the Senate webpage; the law will be called Ley Federal para Prevenir y Sancionar los Delitos Inform ticos (Federal law to prevent and punish informatic felonies<) A bad name to start with, as there are many laws already in that contested area. I started reading with the preamble (Exposici n de motivos), which already shows bad signs of imprecise redaction and is plagued with factual errors (i.e. asserting that the real danger stems from the Web migrating to the Web 2.0, from which stems that this migration and not any previous one. Or by stating that (quoting+translating a full paragraph):
Activities such as electronic commerce, digital periodism, publicity and the opinions, messages or elements written in social networks can lead to patrimonial, reputation, honor or professional activity losses for people.
He continues by stating that only 16% of the countries have some kind of cybersecurity strategy (and, of course, Mexico doesn't). That... Well, is very hard to believe, as Mexico has two separate policial groups devoted to cybersecurity, and laws regulating from electronic signatures, commerce, identity, privacy, use and abuse, and a long list. Of course, as most law proposals go, it quickly decays into a dry, boring document... And I must admit I didn't fully read it, but picked here and there. I won't copy in full the note I mentioned at the beginning at r3d.mx, but will continue with some strange points, such as:
Article 16
Every person that, without the corresponding authorization or exceeding the authorization confered, accesses, intercepts, interfers or uses an information system, will be punished by one to eight years of prision and fined by 800 to 1000 days of minimum wage
So, yes, borrowing your computer without getting explicit permission, or playing around with the options in kiosks, or tons of whatever we curious people do with systems we encounter are basis for jail. (And yes, fines in this country are expressed in "days of minimum wage", which goes at ~MX$70 per day, which is ~US$4). But it gets funkier quickly:
Article 17
Whoever fraudulently destroys, disables, damages or in any way alters the working of an informatic system or any of its components, will be punished by fice to fifteen years of jail and fined by up to a thousand minimum wage days The same punishment will be given to whoever, without authorization, destroys, damages, modifies, divulges, transfers or disables information contained in any Informatic System or any of its components. The punishment will be ten to twenty years in prision and a fine of up to a thousand days of minimum wage if the effects here mentioned are done by the creation, introduction or fraudulent transmission, by any means, of an informatic weapon or malicious code
This law is meant to protect against cyberfelonies, if such a thing exists. However, here we are putting at risk people even for accidental equipment destructions. I dropped your portable hard disk with my elbow off the table? Accuse me of acting fraudulently, and I'm up for a serious jail time. And yes, laws are meant to be interpreted... And I don't want to be at the receiving end of this one! In this last article, Fayad mentions informatic weapons, which are defined in the preamble as any informatic program, informatic system, or in general, any device or material created or designed with the purpose of committing an informatic crime. So the very next article makes me, as it should make all of my fellow students and researchers, very uneasy:
Article 18
Whoever uses informatic weapons or malicious code will be imprisioned by two to six years, and fined with 200 to 500 days of minimum wage.
Article 19
Whoever builds, distributes, commerces with informatic weapons or malicious codes will be punished by three to seven years of prision and 200 to 500 days of minimum wage.
If we need to analyze malware for our classes (or for paid work, or as a hobby), we clearly fall in article 18. If we write something that can be classified as malware (without even releasing it, as an academic excercise only!), we are covered by article 19. If I give my students code that's known to be malicious (which could be as inofensive as linking to a well-known Web comic), I'm also covered by article 19. I'll jump all the way to article 31 (reproduced only partially):
Article 31
Whoever, by any means, creates, captures, records, copies, alters, duplicates, clones or deletes the information contained in a credit or debit card (...) will be punished by 8 to 14 years of prision and 300 to 500 days of minimum wage. (...)
This clearly disincentivates any way of e-commerce. When I try to buy anything online, I have to capture+copy my (rightfully owned) credit card data. The services provider has to copy, process and then delete said information. Any e-transaction is punished by jail! Well... But thinking about this again, maybe I shouldn't be so worried about the malware distribution issue at my classes. There are clearer and more contundent articles. Say...
Article 35
Whoever convenes, organizes, is part of, or executes a cibernetic attack, will be punished by 20 to 30 years of prision and fined with 100 to 1000 days of minimum wage
Of course we have convened, organized, been part of and executed cibernetic attacks at the computer security lab at ESIME. Why would there be such a lab otherwise? Then, there are clear indications that the Senator didn't understand the topic his team was working on:
Article 37
Who manipulates the digital seals used by command of the public authority will be punished with 240 days of community work
Now... What is a digital seal? It's not a phisical one that does not allow opening the doors to a business found at fault, but something that just proves a document is legitimate and pristine. How can I manipulate them? Of course, if the seals are MD5-based, I can easily forge them (and SHA1-based, it seems they will be broken enough soon to be considered no longer trustable)... But that's about it! And there is more, lots more. I'm swamped with work, and have to get back to it. But chapters the following chapters have a lot of potential for finding holes. PS - And yes, the only use I do of Twitter is via the headlines in my blog ;-) [update] Ley Fayad is dead, yay! \o/ The senator withdrew the proposal.

Sven Hoexter: dh, dpkg-buildflags, LDFLAGS and the ugly way to stop overlinking

I just got asked by someone I help out with sponsoring uploads from time to time how to get rid of overlinking. Since wheezy or maybe even earlier dpkg-shlipdeps will complain with a warning like this:
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/
foobar/usr/bin/foobar was not linked against libatk-1.0.so.0 (it uses none of the
library's symbols)
That usually requires some build system fixes, or pkg-config fixes or something else. This one should provide a starting point if you'd like to read a bit more. But in case you're lucky, and you use debhelper, and the buildsystem is sane enough to read the LDCONFIG environment variable
export DEB_LDFLAGS_MAINT_PREPEND := -Wl,-z,defs -Wl,--as-needed
in your debian/rules can be enough. I admit that's still a bit of a hack that requires some luck to work out and can break in strange and nasty ways. But if it work's I'm fine with it. man 1 dpkg-buildflags hold the details to the mechanics in use here regarding the environment variables.

Andrew Cater: MiniDebconf2015, ARM Cambridge 1320 5 November

Back after lunch - chatting about video and stuff with the Video Team sprint.
Thanks to the ARM employees swiping us in and out and shepherding the bunch of us over to lunch and keeping track of us.

From behind me, there's the chat over sorting out a switch to handle the LAN setup to test the video equipment later on.

Next door, they're busy, busy, busy - four tables full of people working fairly quietly. Steve McIntyre (Sledge) is running around facilitating, answering questions, sorting out "stuff"

Front desk are still front desking although most of us have arrived: I'm guessing they'll be busiest over the weekend.

Three of us on laptops doing whatever in here: Helen busy working to make sense of scribbled notes on whiteboards into a proper diagramming package - green and red marker pen on two walls full of whiteboard becomes a sensible project - objectives, bullet points, tasking - seemingly far too much organisation for this sort of thing and not the way some of us naturally work but necessary if we're not to forget something tiny but vital.

Thanks for this expertise, Helen, that most of us don't have :)


Lars Wirzenius: Obnam 1.18 released (backup software)

I have just released version 1.18 of Obnam, my backup program. See the website at http://obnam.org for details on what the program does. The new version is available from git (see http://git.liw.fi) and as Debian packages from http://code.liw.fi/debian, and uploaded to Debian, and soon in unstable. The NEWS file extract below gives the highlights of what's new in this version. Version 1.18, released 2015-11-04 Bug fixes: Minor changes:

4 November 2015

Mart n Ferrari: Tales from the SRE trenches: What can SRE do for you?

This is the fourth part in a series of articles about SRE, based on the talk I gave in the Romanian Association for Better Software. Previous articles can be found here: part 1, part 2, and part 3. As a side note, I am writing this from Heathrow Airport, about to board my plane to EZE. Buenos Aires, here I come!

So, what does SRE actually do? Enough about how to keep the holy wars under control and how to get work away from Ops. Let's talk about some of the things that SRE does. Obviously, SRE runs your service, performing all the traditional SysAdmin duties needed for your cat pictures to reach their consumers: provisioning, configuration, resource allocation, etc. They use specialised tools to monitor the service, and get alerted as soon as a problem is detected. They are also the ones waking up to fix your bugs in the middle of the night. But that is not the whole story: reliability is not only impacted by new launches. Suddenly usage grows, hardware fails, networks lose packets, solar flares flip your bits... When working at scale, things are going to break more often than not. You want these breakages to affect your availability as little as possible, there are three strategies you can apply to this end: minimise the impact of each failure, recover quickly, and avoid repetition. And of course, the best strategy of them all: preventing outages from happening at all.

Minimizing the impact of failures A single failure that takes the whole service down will affect severely your availability, so the first strategy as an SRE is to make sure your service is fragmented and distributed across what is called "failure domains". If the data center catches fire, you are going to have a bad day, but it is a lot better if only a fraction of your users depend on that one data center, while the others keep happily browsing cat pictures. SREs spend a lot of their time planning and deploying systems that span the globe to maximise redundancy while keeping latencies at reasonable levels.

Recovering quickly Many times, retrying after a failure is actually the best option. So another strategy is to automatically restart in milliseconds any piece of your system that fails. This way, less users are affected, while a human has time to investigate and fix the real problem. If a human needs to intervene, it is crucial that they get notified as fast as possible, and that they have quick access to all the information that is needed to solve the problem: detailed documentation of the production environment, meaningful and extensive monitoring, play-books1, etc. After all, at 2 AM you probably don't remember in which country the database shard lives, or what were the series of commands to redirect all the traffic to a different location. Implementing monitoring and automated alerts, writing documentation and play-books, and practising for disaster are other areas where SREs devote much effort.

Avoiding repetition Outages happen, pages ring, problems get fixed, but it should always be a chance for learning and improving. A page should require a human to think about a problem and find a solution. If the same problem keeps appearing, the human is not needed any more: the problem needs to be fixed at the root. Another key aspect of dealing with incidents is to write post-mortems2. Every incident should have one, and these should be tools for learning, not finger-pointing. Post-mortems can be an excellent tool, if people are honest about their mistakes, they are not used to blame other people, and issues are followed up by bug reports and discussions.

Preventing outages Of course nobody can prevent hard drives from failing, but there are certain classes of outages that can be forecasted with careful analysis. Usage spikes can bring a service down, but an SRE team will ensure that the systems are load-tested at higher-than-normal rates. They could also be prepared to quickly scale the service, provided the monitoring system will alert as soon as a certain threshold is reached. Monitoring is actually the key part in this: measuring relevant metrics for all the components of a system, and following their trends over time, SREs can completely avoid many outages: latencies growing out of acceptable bounds, disks filling up, progressive degradation of components, are all examples of typical problems automatically monitored (and alerted on) in an SRE team.
This is getting pretty long, so the next post will be the last one of these series, with some extra tips from SRE experience that I am sure can be applied in many places.

  1. A play-book is a document containing critical information needed when dealing with a specific alert: possible causes, hints for troubleshooting, links to more documentation. Some monitoring systems will automatically add a play-book URL to every alert sent, and you should be doing it too.
  2. Similarly to their real-life counterparts, a post-mortem is the process of examining a dead body (the outage), gutting it out and trying to understand what caused its demise.
Comment

Yves-Alexis Perez: Uploading source-only packages built with pbuilder

Thanks to Mehdi Dogguy, here's a nice hook to generate a source change file at build time (with pbuilder), so one can upload source-only packages to the archive and have buildds rebuild for all the architectures. Put it in .pbuilder/hooks/B10_source-build so it gets called once the builds succeeds
#! /bin/sh
generate_change_file()
 
  local version=$(dpkg-parsechangelog -Sversion)
  local package=$(dpkg-parsechangelog -Ssource)
  echo "Generating source changes file"
  dpkg-genchanges -S > ../$ package _$ version _source.changes
 
cd /tmp/buildd/*/debian/..
generate_change_file
Next time you build a package, you should find, alongside the <package>_<version>_<arch>.changes file, a <package>_<version>_source.changes which you can use with usual tools (lintian, debsign, dput ) to upload it to the Debian archive. Note that if you do that, you have to make sure that your debian/rules support building separately the arch-dependent and arch-independant packages. To check that, you can call pdebuild like this:
pdebuild --debbuildopts -A # binary-only build, limited to arch-independant packages
pdebuild --debbuildopts -B # binary-only build, limited to arch-dependant packages

Arturo Borrero Gonz lez: Rant: Software Engineering in the University of Seville



I'm an IT engineering student at the University of Seville, where there are 3 flavours to choose:
* Computers engineering: mostly focused on hardware
* Software engineering: focused on software
* IT engineering: a mixture of things, including subjects dedicated to security and virtualization

Each of these last a total amount of 4 years (better said: courses).

Software Engineering is mostly the traditional IT engineering and it's what I'm doing.

The first course is almost common to all engineering: algebra, calculus, phyisics, electronics and so on.
You may find these subjects in all IT engineering flavours and also in other traditional engineering
like telecommunications, mechanical, industrial or aeronautics.

In the second year, the subjects tend to be specific for IT engineering: networking, databases, algorithms and so on.
The third and fourth courses are almost fully specific for each flavour of IT engineering.

The problem I've found is in the third year of Software engineering.
We have here subjects like the following:
* requirements engineering
* design and testing
* software process management

All of these are focused in what they consider our biggest possibility in our professional career: the regional government.

It turns out this is not Silicon Valley, and the regional government (Junta de Andaluc a) is in fact the biggest IT player in our region.
They run thousands of servers, networks and digital services both for the public and for internal administrative usage.
The government is known for massively running concrete technologies: Oracle and java. Almost all of the biggest applications are running these technologies.

The problem here is that our subjects are fully headed towards these technologies, also using methodologies
that I consider outdated. They think you are going to work in a classical software factory on the classical software consulting business.
And they do this explicitly: the first day in the 'design & testing' subject we were told by the professor that we are likely going to work in the 'local industry', so we need to learn these technologies and no others.

An example: last week I was given by a professor a VirtualBox (sic) virtual machine running Windows XP (yeah...) just to run Eclipse (for java development) in a controlled environment. They expect the host OS to be Windows as well.
Other example: to this day, no subject which requires the usage of a VCS is using git. They stick to svn.

This is very sad!

The other day, it felt event worse because a RedHat internship was announce in a college mailing list, which explicitly excluded students from Software Engineering.... because all the reasons above. How is this possible? RedHat is a software company!

Let's be constructive. These are some things I would like to see in Software Engineering:
* other software workflows (for example, the linux kernel development model or debian package lifecycle)
* modern free libre open source technologies! python! golang!
* packaging (for example: debian, rpm, docker...)
* new database systems (nosql) or at least no more Oracle (please switch to PostgreSQL or whatever)
* no more java! enough of java!

BTW some time ago I read somewhere that no student should graduate without contributing to a major open source project. I can't agree more.

Rapha&#235;l Hertzog: My Free Software Activities in October 2015

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 13.25 hours on Debian LTS. During this time I worked on the following things: I also started a conversation about what paid contributors could work on if they have some spare cycle as the current funding level might allow us to invest some time on work outside of just plain security updates. The Debian Administrator s Handbook I spent quite some time finalizing the Jessie book update, both for the content and for the layout of the printed book. Debian Handbook: cover of the jessie edition Misc Debian work GNOME 3.18. I uploaded a new gnome-shell-timer working with GNOME Shell 3.18 and I filed bugs #800660 and #802480 about an annoying gnome-keyring regression I did multiple test rounds with the Debian maintainers (Dmitry Shachnev, kudos to him!) and the upstream developers (see here and here). Apart from those regressions, I like GNOME 3.18! Python-modules team migration to Git. After the Git migration, and since the team policy now imposes usage of git-dpm on all members, I made some tries with it on the python-django package while pushing version 1.8.5 to experimental. And the least I can say is that I m not pleased with the result. I thus filed 3 bugs summarizing the problems I have with git-dpm: #801666 (no way to set the upstream branch names from within the repository), #801667 (no clean way to merge between packaging branches), #801668 (does not create upstream tag immediately on tarball import). That is on top of other randomly stupid bugs that were already reported like #801548 (does not work with perfectly valid pre-existing upstream tags). Django packaging. I filed bugs on all packages build-depending on python-django that fail to build with Django 1.8 and informed them that I would upload Django 1.8 to unstable in early November (it s done already). Then I fixed python-django-jsonfield myself since Distro Tracker relies on this package. Following this small mass-bug filing, I filed a wishlist bug on devscripts to improve the mass-bug helper script (see #801926). And since I used ratt to rebuild the packages, I filed a wishlist issue on this new tool as well. Tryton 3.6 upgrade. I upgraded my own Tryton installation to version 3.6 and filed bug #803066 because the SysV init script was not working properly. That also reminded me that the DD process of Matthias Behrle (the tryton package maintainer) was stalled due to a bug in the NM infrastructure so I pinged the NM team and we sorted out a way for me to advocate him and get his process going Distro Tracker. I continued my work to refactor the way we handle incoming mail processing (branch people/hertzog/mailprocessing). It s now mostly finished and I need to deploy it in a test environment before being able to roll it out on tracker.debian.org. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Vincent Sanders: I am not a number I am a free man

Once more the NetSurf developers tried to escape from a mysterious village by writing web browser code.

Michael Drake, Daniel Silverstone, Dave Higton and Vincent Sanders at NetSurf Developer workshop
The sixth developer workshop was an opportunity for us to gather together in person to contribute to NetSurf.

We were hosted by Codethink in their Manchester offices which provided a comfortable and pleasant space to work in.

Four developers managed to attend in person from around the UK: Michael Drake, Daniel Silverstone, Dave Higton and Vincent Sanders.

The main focus of the weekends activities was to work on improving our JavaScript implementation. At the previous workshop we had laid the groundwork for a shift to the Duktape JavaScript engine and since then put several hundred hours of time into completing this transition.

During this weekend Daniel built upon this previous work and managed to get DOM events working. This was a major missing piece of implementation which will mean NetSurf will be capable of interpreting JavaScript based web content in a more complete fashion. This work revealed several issues with our DOM library which were also resolved.

We were also able to merge several improvements provided by the Duktape upstream maintainer Sami Vaarala which addressed performance problems with regular expressions which were causing reports of "hangs" on slow processors.

The responsiveness of Sami and the Ducktape project has been a pleasant surprise making our switch to the library look like an increasingly worthwhile effort.

Overall some good solid progress was made on JavaScript support. Around half of the DOM interfaces in the specifications have now been implemented leaving around fifteen hundred methods and properties remaining. There is an aim to have this under the thousand mark before the new year which should result in a generally useful implementation of the basic interfaces.

Once the DOM interfaces have been addressed our focus will move onto the dynamic layout engine necessary to allow rendering of the changing content.

The 3.4 release is proposed to occur sometime early in the new year and depends on getting the JavaScript work to a suitable stable state.

Dave joined us for the first time, he was principally concerned with dealing with bugs and the bug tracker. It was agreeable to have a new face at the meeting and some enthusiasm for the RISC OS port which has been lacking an active maintainer for some time.

The turnout for this workshop was the same as the previous one and the issues raised then are still true. We still have a very small active core team who can commit only limited time which is making progress very slow and are lacking significant maintenance for several frontends.

Overall we managed to pack 16 hours of work into the weekend and addressed several significant problems.

Next.

Previous.