Search Results: "Konstantinos Margaritis"

31 March 2015

Konstantinos Margaritis: "Advanced Java EE Development with WildFly" released by Packt (I was one of the reviewers!)

For the past months I had the honour and pleasure of being one of the reviewers of "Advanced Java EE Development with WildFly" by Deepak Vohra. Today, I'm pleased to announce that the book has just been released by Packt: https://www.packtpub.com/application-development/advanced-java-ee-development-wildfly It was my first time being a reviewer and it was a very interesting experience. I would like to thank the two Project Coordinators from Packt, Aboli Ambardekar and Suzanne Coutinho, who guided me with the reviewing process, so that my review would be as accurate as possible and only related to technical aspect of the book. Looking at the process retrospectively I now begin to understand the complication of achieving a balance between the author's vision for the book and the scrutiny of the (many) reviewers. And of course I would like to thank the author, Deepak Vohra, for writing the book in the first place, I'm looking forward to reading the actual physical book :)

10 November 2014

Konstantinos Margaritis: New owner/maintainer for CSVChart Drupal module

Back in ~2008, I had created a small Drupal module that used CustomFilter and Google Chart Tools: Image Charts, called CSV Chart. It was a simple module that took embedded CSV data and presented them as Google charts. It was very handy for what I wanted -and still want- to do, present benchmarks for my work on Altivec (and now in general SIMD). However, I don't really code in PHP anymore -haven't coded PHP since 2010- so the module was left to bitrot. I did some minor adjustment to run with the current D7-based site, but that was it. Thankfully, others found the module useful to use and adopt and the result has been that in the last days I transfered ownership of the CSV Chart to Pierre Vriens, so that he can continue development and maintenance of this tiny module. I would like to publicly thank Pierre for his work! And that's the beauty of Free Software!

31 October 2014

Konstantinos Margaritis: SIMD on javascript, MHO

Just read the Mozilla announcement on SIMD.js and I can say I got mixed feelings about this. I don't really comment other news/blogs/announcements, but this is an exception. On one hand, I definitely welcome more SIMD use everywhere, being a SIMD advocate and enthusiast for many years (since 2004 actually). So seeing more of it and by someone such as Mozilla, that's even better! On the other hand, wait, that's SIMD in Javascript?!? Really? Why? As if we already covered native coverage of SIMD in every other part, including the browser itself (No browser uses SIMD extensively in its core, though that would prove to be of actual benefit, the only SIMD code I know of is in the media playback code, which is usually some external library like ffmpeg/x264/etc that already has SIMD optimized parts anyway. So, instead of using resources to optimize the core browser with SIMD -I'm sure there will be plenty of opportunities in the codebase for such optimizations- so that every web application, including Javascript ones will be optimized, or even the JS JITs themselves, yet Mozilla wants to push the effort to the web app developer to use their SIMD.js to do the equivalent of what SIMD coders have been doing to native apps for a long time now, only for JS apps. Ok, so what's the gain? I read the PDF presentation that shows mandebrot.js to go from 9 FPS to 37FPS using SIMD.js. Admittedly that's impressive. But it also proves that the whole buzz about lower energy footprint computers, power efficiency is just useless. Why is that? For comparison running Xaos (fractal/mandelbrot program) on my very low end PC (2-core Athlon X2, AM2 socket, so DDR2) gives me ~250FPS, and I'm not even sure it's using SSE at all (from a simple check it doesn't). Zooming is realtime and at full detail. In the same talk, there was a benchmark of LLVM Javascript being as fast as C++ or 1.5x the native running time. I admit haven't tried the tests listed, but the mandelbrot test was using asm.js and 9 is definetely not 250/1.5. But I guess I'm just picky. So, the latest trend of moving everything to the browser and JS,means that instead of optimizing my apps to run great on native, instead of making stuff running faster on my 5Wt big.Little 8-core ARM SoC, I have to get a much more power-hungry CPU to see the same performance. I'm totally against that. I want my newer CPUs, who are more energy-efficient and faster to actually feel like THAT. I don't want to upgrade just to experience the performance of a 486 20 years ago! The talk mentioned HTML5 (and hence javascript) overtaking all other platforms for application development everywhere, including the smartphones. I certainly hope that's not the case, and I know of many people who also don't feel that way. We're not buying the "Everything on the web/cloud" paradigm, but I guess we're just a minority. I could go on for a long time, but I have an actual SIMD-related bug to fix, cheers. Note: I used to have comments enabled on my blog, but moderating spam was too time consuming, even with CAPTCHAs, so I disabled them entirely, if anyone could suggest of a better method, I'd gladly take advice -have been thinking about disqus, not sure if it's actually a good solution).

22 October 2014

Konstantinos Margaritis: Eigen NEON port extended to ARMv8!

Soon after the VSX port, and as promised I have completed the ARMv8 NEON (a.k.a. Advanced SIMD) port. Basically this extends support to 64-bit doubles and also provides faster alternatives to division as ARMv8 has builtin instructions for division both for 32-bit floats and 64-bit doubles. Preliminary benchmarks (bench_gemm):

13 October 2014

Konstantinos Margaritis: SIMD optimizations, cont.

A friend of mine told me that I should advertise my passion and know-how about SIMD more, and I decided to follow his advice. Though I am terrible at marketing and even more at personal marketing, I've made an attempt to do just that, advertise the fact that I'm offering SIMD Optimization Services (with emphasis on PowerPC AltiVec/VMX/VSX, and ARM NEON, but I'm ok with SSE as well, the logic is pretty much the same, though the difference(s) are in the details). For this reason I'm offering a free evaluation of your performance critical code (open/closed, able to sign NDAs if needed) to let you know if it's worth optimizing it, what kind of a performance gain you would get and how much it would cost you to get that result.
You can read more here.

22 September 2014

Konstantinos Margaritis: EfikaMX updated wheezy and jessie images available

A while ago, I promised to some people in powerdeveloper.org forum that I would provide bootable armhf images for wheezy but most importantly for jessie with an updated kernel. After a delay -I did have the images ready and working, but had to clean them up a bit- I decided to publish them here first. So, here are the images: http://freevec.org/files/efikamx-wheezy-armhf-20140921.img.xz (559MB)
http://freevec.org/files/efikamx-jessie-armhf-20140921.img.xz (635MB)

21 September 2014

Konstantinos Margaritis: VSX port added to Eigen!

Being the SIMD fanatic that I am, a few years ago I did the PowerPC Altivec and ARM NEON port for the Eigen linear algebra library, one of the best and most popular libraries -and most ported. Recently I thought it would be a good idea to extend both ports to 64-bit, and it would also help me with the SIMD book, using VSX in one case and ARMv8 NEON (or Advanced SIMD as ARM likes to call it) in the latter. ARMv8 hardware is a bit scarce at the moment, so I thought I'd start with VSX. Being in Debian, I have access to a number of porterboxes in several architectures, and luckily one of those was a Power7 (with VSX) running ppc64. So I started the porting -or rather extending the code- to use VSX in the 64-bit doubles case. Unluckily, I could not test anything because Debian kernels do not have VSX enabled in wheezy -which is what the porterbox is running and enabling it is a non-option(#758620). So, running VSX code would turn out to be quite hard.

4 August 2014

Konstantinos Margaritis: SIMD book, "Sponsored by ARM"!

Ok, took a while but I got the final word about this and can announce that the sponsor who donated 500 EUR to the Indiegogo campaign was ARM itself! I have to thank my friends at ARM@Cambridge and especially Dr Monika Biddulph, General Manager, Partner Enablement Group at ARM. When the book goes to print you can be sure it will include "Sponsored by ARM" somewhere! :) Also a friendly reminder that even if the campaign is over, I still welcome the support in the form of preorders/sponsorships.

30 July 2014

Konstantinos Margaritis: SIMD book, update 3! Addition/Subtraction mostly finished

Finally. Apologies for the delay, but it's been a busy month. This time I will hold true to my word and upload a PDF for people to see (attached to this page). So, what's new? Here is a list of things done: * Finished ALL addition/subtraction related instructions for all engines and major derivatives (SSE*/AVX, VMX/VSX, NEON/armv8 NEON). With diagrams (these were the reasons it has taken so long).
* Reorganized the structure (split the book into Parts I/II, the instruction index will be in Part II, Part I will carry the design analysis of each SIMD engine.
* Added an TOC/index.
* So far, with just Addition/Subtraction Chapter and the rest empty sections, it has reached 175 pages (B5, again I'm not fixed on the size, it might actually change)! My estimate is that the whole book will surpass 800 pages with everything included. TODO:

9 June 2014

Konstantinos Margaritis: SIMD book, second update!

From the Indiegogo page: https://www.indiegogo.com/projects/simd-engines-comparative-reference-guide/x/4966960#activity
  • Added titlepage (simple, but will do the job)
  • Reorganized ALL instructions to include both unsigned/signed in the same entity
  • Added Saturated Addition, Subtraction and Saturated Subtraction
  • Added ARMv8 NEON instructions taken from ARM infocenter draft
  • Fixed some instructions (added 64-bit arithmetic for NEON)
  • Added some special addition/subtraction, like add/sub with carry(vmx/vsx), addsub(SSE3/AVX)
  • Added some in-vector sum additions, sum reductions but no descriptions yet
  • Added diagram for 8-bit addition/subtraction (still need lots more).
  • Removed VMX128, couldn't find enough information, an email to some IBM toolchain developers was left unanswered, so I guess noone really will really care if that engine is left out, if enough people insist on it, please also be kind enough to provide some documentation on it.

3 June 2014

Konstantinos Margaritis: SIMD book, first draft published!

Check activity here: https://www.indiegogo.com/projects/simd-engines-comparative-reference-guide/x/4966960#activity From the update: Ok, I've been busy the past days, I started writing the book (using LaTeX :), and I'd like to say that progress has been good. I fixed the current list of SIMD engines that I'm going to include and it's a long one:

15 June 2011

Christian Perrier: So, what happened with Kikithon?

I mentioned this briefly yesterday, but now I'll try to summarize the story of a great surprise and a big moment for me. All this started when my wife Elizabeth and my son Jean-Baptiste wanted to do something special for my 50th birthday. So, it indeed all started months ago, probably early March or something (I don't yet have all the details). Jean-Baptiste described this well on the web site, so I won't go again into details, but basically, this was about getting birthday wishes from my "free software family" in, as you might guess, as many languages as possible. Elizabeth brought the original idea and JB helped her by setting up the website and collecting e-mail addresses of people I usually work with: he grabbed addresses from PO files on Debian website, plus some in his own set of GPG signatures and here we go. And then he started poking dozens of you folks in order to get your wishes for this birthday. Gradually, contributions accumulated on the website, with many challenges for them: be sure to get as many people as possible, poking and re-poking all those FLOSS people who keep forgetting things... It seems that poking people is something that's probably in the Perrier's genes! And they were doing all this without me noticing. As usually in Debian, releasing on time is a no-no. So, it quickly turned out that having everything ready by April 2nd wouldn't be possible. So, their new goal was offering this to me on Pentecost Sunday, which was yesterday. And...here comes the gift. Aha, this looks like a photo album. Could it be a "50 years of Christian" album? But, EH, why is that pic of me, with the red Debconf5 tee-shirt (that features a world map) and a "bubulle" sign, in front of the book? But, EH EH EH, what the .... are doing these word by H0lger, then Fil, then Joey doing on the following pages? And only then, OMG, I discover the real gift they prepared. 106, often bilingual, wishes from 110 people (some were couples!). 18 postcards (one made of wood). 45 languages. One postcard with wishes from nearly every distro representatives at LinuxTag 2011. Dozens of photos from my friends all around the world. All this in a wonderful album. I can't tell what I said. Anyway, JB was shooting a video, so...we'll see. OK, I didn't cry...but it wasn't that far and emotion was really really intense. Guys, ladies, gentlemen, friends....it took me a while to realize what you contributed to. It took me the entire afternoon to realize the investment put by Elizabeth and JB (and JB's sisters support) into this. Yes, as many of you wrote, I have an awesome family and they really know how to share their love. I also have an awesome virtual family all around the world. Your words are wholeheartedly appreciated and some were indeed much much much appreciated. Of course, I'll have the book in Banja Luka so that you can see the result. I know (because JB and Elizabeth told me) that many of you were really awaiting to see how it would be received (yes, that includes you, in Germany, who I visited in early May!!!). Again, thank you so much for this incredible gift. Thank you Holger Levsen, Phil Hands, Joey Hess, Lior Kaplan, Martin Michlmayr, Alberto Gonzalez Iniesta, Kenshi "best friend" Muto, Praveen Arimbrathodiyil, Felipe Augusto van de Wiel, Ana Carolina Comandulli (5 postcards!), Stefano Zacchiroli (1st contribution received by JB, of course), Gunnar Wolf, Enriiiiiico Zini, Clytie Siddall, Frans Pop (by way of Clytie), Tenzin Dendup, Otavio Salvador, Neil McGovern, Konstantinos Margaritis, Luk Claes, Jonas Smedegaard, Pema Geyleg, Meike "sp tzle queen" Reichle, Alexander Reichle-Schmehl, Torsten Werner, "nette BSD" folks, CentOS Ralph and Brian, Fedora people, SUSE's Jan, Ubuntu's Lucia Tamara, Skolelinux' Paul, Rapha l Hertzog, Lars Wirzenius, Andrew McMillan (revenge in September!), Yasa Giridhar Appaji Nag (now I know my name in Telugu), Amaya Rodrigo, St phane Glondu, Martin Krafft, Jon "maddog" Hall (and God save the queen), Eddy Petri or, Daniel Nylander, Aiet Kolkhi, Andreas "die Katze geht in die K che, wunderbar" Tille, Paul "lets bend the elbow" Wise, Jordi "half-marathon in Banja Luka" Mallach, Steve "as ever-young as I am" Langasek, Obey Arthur Liu, YAMANE Hideki, Jaldhar H. Vyas, Vikram Vincent, Margarita "Bronx cross-country queen" Manterola, Patty Langasek, Aigars Mahinovs (finding a pic *with* you on it is tricky!), Thepittak Karoonboonyanan, Javier "nobody expects the Spanish inquisition" Fern ndez-Sanguino, Varun Hiremath, Moray Allan, David Moreno Garza, Ralf "marathon-man" Treinen, Arief S Fitrianto, Penny Leach, Adam D. Barrat, Wolfgang Martin Borgert, Christine "the mentee overtakes the mentor" Spang, Arjuna Rao Chevala, Gerfried "my best contradictor" Fuchs, Stefano Canepa, Samuel Thibault, Eloy "first samba maintainer" Par s, Josip Rodin, Daniel Kahn Gillmor, Steve McIntyre, Guntupalli Karunakar, Jano Gulja , Karolina Kali , Ben Hutchings, Matej Kova i , Khoem Sokhem, Lisandro "I have the longest name in this list" Dami n Nicanor P rez-Meyer, Amanpreet Singh Alam, H ctor Or n, Hans Nordhaugn, Ivan Mas r, Dr. Tirumurti Vasudevan, John "yes, Kansas is as flat as you can imagine" Goerzen, Jean-Baptiste "Piwet" Perrier, Elizabeth "I love you" Perrier, Peter Eisentraut, Jesus "enemy by nature" Climent, Peter Palfrader, Vasudev Kamath, Miroslav "Chicky" Ku e, Mart n Ferrari, Ollivier Robert, Jure uhalev, Yunqiang Su, Jonathan McDowell, Sampada Nakhare, Nayan Nakhare, Dirk "rendez-vous for Chicago marathon" Eddelbuettel, Elian Myftiu, Tim Retout, Giuseppe Sacco, Changwoo Ryu, Pedro Ribeoro, Miguel "oh no, not him again" Figueiredo, Ana Guerrero, Aur lien Jarno, Kumar Appaiah, Arangel Angov, Faidon Liambotis, Mehdi Dogguy, Andrew Lee, Russ Allbery, Bj rn Steensrud, Mathieu Parent, Davide Viti, Steinar H. Gunderson, Kurt Gramlich, Vanja Cvelbar, Adam Conrad, Armi Be irovi , Nattie Mayer-Hutchings, Joerg "dis shuld be REJECTed" Jaspert and Luca Capello. Let's say it gain:

13 June 2011

Christian Perrier: So, what happened with Kikithon?

I mentioned this briefly yesterday, but now I'll try to summarize the story of a great surprise and a big moment for me. All this started when my wife Elizabeth and my son Jean-Baptiste wanted to do something special for my 50th birthday. So, it indeed all started months ago, probably early March or something (I don't yet have all the details). Jean-Baptiste described this well on the web site, so I won't go again into details, but basically, this was about getting birthday wishes from my "free software family" in, as you might guess, as many languages as possible. Elizabeth brought the original idea and JB helped her by setting up the website and collecting e-mail addresses of people I usually work with: he grabbed addresses from PO files on Debian website, plus some in his own set of GPG signatures and here we go. And then he started poking dozens of you folks in order to get your wishes for this birthday. Gradually, contributions accumulated on the website, with many challenges for them: be sure to get as many people as possible, poking and re-poking all those FLOSS people who keep forgetting things... It seems that poking people is something that's probably in the Perrier's genes! And they were doing all this without me noticing. As usually in Debian, releasing on time is a no-no. So, it quickly turned out that having everything ready by April 2nd wouldn't be possible. So, their new goal was offering this to me on Pentecost Sunday, which was yesterday. And...here comes the gift. Aha, this looks like a photo album. Could it be a "50 years of Christian" album? But, EH, why is that pic of me, with the red Debconf5 tee-shirt (that features a world map) and a "bubulle" sign, in front of the book? But, EH EH EH, what the .... are doing these word by H0lger, then Fil, then Joey doing on the following pages? And only then, OMG, I discover the real gift they prepared. 106, often bilingual, wishes from 110 people (some were couples!). 18 postcards (one made of wood). 45 languages. One postcard with wishes from nearly every distro representatives at LinuxTag 2011. Dozens of photos from my friends all around the world. All this in a wonderful album. I can't tell what I said. Anyway, JB was shooting a video, so...we'll see. OK, I didn't cry...but it wasn't that far and emotion was really really intense. Guys, ladies, gentlemen, friends....it took me a while to realize what you contributed to. It took me the entire afternoon to realize the investment put by Elizabeth and JB (and JB's sisters support) into this. Yes, as many of you wrote, I have an awesome family and they really know how to share their love. I also have an awesome virtual family all around the world. Your words are wholeheartedly appreciated and some were indeed much much much appreciated. Of course, I'll have the book in Banja Luka so that you can see the result. I know (because JB and Elizabeth told me) that many of you were really awaiting to see how it would be received (yes, that includes you, in Germany, who I visited in early May!!!). Again, thank you so much for this incredible gift. Thank you Holger Levsen, Phil Hands, Joey Hess, Lior Kaplan, Martin Michlmayr, Alberto Gonzalez Iniesta, Kenshi "best friend" Muto, Praveen Arimbrathodiyil, Felipe Augusto van de Wiel, Ana Carolina Comandulli (5 postcards!), Stefano Zacchiroli (1st contribution received by JB, of course), Gunnar Wolf, Enriiiiiico Zini, Clytie Siddall, Frans Pop (by way of Clytie), Tenzin Dendup, Otavio Salvador, Neil McGovern, Konstantinos Margaritis, Luk Claes, Jonas Smedegaard, Pema Geyleg, Meike "sp tzle queen" Reichle, Alexander Reichle-Schmehl, Torsten Werner, "nette BSD" folks, CentOS Ralph and Brian, Fedora people, SUSE's Jan, Ubuntu's Lucia Tamara, Skolelinux' Paul, Rapha l Hertzog, Lars Wirzenius, Andrew McMillan (revenge in September!), Yasa Giridhar Appaji Nag (now I know my name in Telugu), Amaya Rodrigo, St phane Glondu, Martin Krafft, Jon "maddog" Hall (and God save the queen), Eddy Petri or, Daniel Nylander, Aiet Kolkhi, Andreas "die Katze geht in die K che, wunderbar" Tille, Paul "lets bend the elbow" Wise, Jordi "half-marathon in Banja Luka" Mallach, Steve "as ever-young as I am" Langasek, Obey Arthur Liu, YAMANE Hideki, Jaldhar H. Vyas, Vikram Vincent, Margarita "Bronx cross-country queen" Manterola, Patty Langasek, Aigars Mahinovs (finding a pic *with* you on it is tricky!), Thepittak Karoonboonyanan, Javier "nobody expects the Spanish inquisition" Fern ndez-Sanguino, Varun Hiremath, Moray Allan, David Moreno Garza, Ralf "marathon-man" Treinen, Arief S Fitrianto, Penny Leach, Adam D. Barrat, Wolfgang Martin Borgert, Christine "the mentee overtakes the mentor" Spang, Arjuna Rao Chevala, Gerfried "my best contradictor" Fuchs, Stefano Canepa, Samuel Thibault, Eloy "first samba maintainer" Par s, Josip Rodin, Daniel Kahn Gillmor, Steve McIntyre, Guntupalli Karunakar, Jano Gulja , Karolina Kali , Ben Hutchings, Matej Kova i , Khoem Sokhem, Lisandro "I have the longest name in this list" Dami n Nicanor P rez-Meyer, Amanpreet Singh Alam, H ctor Or n, Hans Nordhaugn, Ivan Mas r, Dr. Tirumurti Vasudevan, John "yes, Kansas is as flat as you can imagine" Goerzen, Jean-Baptiste "Piwet" Perrier, Elizabeth "I love you" Perrier, Peter Eisentraut, Jesus "enemy by nature" Climent, Peter Palfrader, Vasudev Kamath, Miroslav "Chicky" Ku e, Mart n Ferrari, Ollivier Robert, Jure uhalev, Yunqiang Su, Jonathan McDowell, Sampada Nakhare, Nayan Nakhare, Dirk "rendez-vous for Chicago marathon" Eddelbuettel, Elian Myftiu, Tim Retout, Giuseppe Sacco, Changwoo Ryu, Pedro Ribeoro, Miguel "oh no, not him again" Figueiredo, Ana Guerrero, Aur lien Jarno, Kumar Appaiah, Arangel Angov, Faidon Liambotis, Mehdi Dogguy, Andrew Lee, Russ Allbery, Bj rn Steensrud, Mathieu Parent, Davide Viti, Steinar H. Gunderson, Kurt Gramlich, Vanja Cvelbar, Adam Conrad, Armi Be irovi , Nattie Mayer-Hutchings, Joerg "dis shuld be REJECTed" Jaspert and Luca Capello. Let's say it gain:

30 April 2011

Thomas Girard: ACE+TAO Debian packaging moved to git

We recently converted Debian ACE+TAO package repository from Subversion to git. This was a long and interesting process; I learned a lot on git in the course. I had been using git for a while for other packages: BOUML, dwarves and GNU Smalltalk. But I did not really get it. A preliminary study led by Pau[1] showed that out of the following three tools: the last one was giving results that look better.
The conversion svn-all-fast-export requires physical access to the repo, so the Alioth SVN repo was copied on my machine svn-pkg-ace/ before running the tool: 
svn-all-fast-export --identity-map authors.txt --rules pkg-ace.rules svn-pkg-ace
Here's the content of the pkg-ace.rules configuration file that was used: 
create repository pkg-ace
end repository
match /trunk/
  repository pkg-ace
  branch master
end match
match /(branches tags)/([^/]+)/
  repository pkg-ace
  branch \2
end match
The author mapping file authors.txt being: 
markos = Konstantinos Margaritis <email-hidden>
mbrudka-guest = Marek Brudka <email-hidden>
pgquiles-guest = Pau Garcia i Quiles <email-hidden>
tgg = Thomas Girard <email-hidden>
tgg-guest = Thomas Girard <email-hidden>
The tool sample configuration file merged-branches-tags.rules recommends to post-process tags, which are just a branch in SVN. That's why the configuration file above treats branches as tags. The conversion was indeed fast: less than 1 minute.
Post-conversion observations Invoking gitk --all in the converted repo revealed different kind of issues:

  • svn tags as branches: http://thomas.g.girard.free.fr/ACE/tags-as-branches.png Branches are marked with green rectangles, and tags with yellow arrows. What we have here (expected given our configuration of the tool) are branches (e.g. 5.4.7-5) corresponding to tags, and tags matching the SVN tagging commit (e.g. backups/5.4.7-5@224). We'll review and fix this.

  • merged code that did not appear as such: http://thomas.g.girard.free.fr/ACE/missing-merge-metadata.png Branches that were not merged using svn merge look like they were not merged at all.

  • commits with wrong author: http://thomas.g.girard.free.fr/ACE/wrong-author.png Before being in SVN, the repository was stored in CVS. When it was imported into SVN, no special attention was given to the commit author. Hence I got credited for changes I did not write.

  • obsolete branches: http://thomas.g.girard.free.fr/ACE/obsolete-branches.png The tool leaves all branches, including removed ones (with tag on their end) so that you can decide what to do with them.

  • missing merges: http://thomas.g.girard.free.fr/ACE/missing-merge.png The branch 5.4.7-12 was never merged into the trunk!

Learning git Based on observations above, I realized my limited knowledge won't do to complete the conversion and clean the repository. There are tons of documentation on git out there, and you can find a lot of links from the git documentation page. Here's the one I've used:
The Git Object Model It's described with pictures here. You really need to understand this if you haven't already. Once you do, you understand that git is built bottom-up: the plumbing then the porcelain. If you can't find the tool you need, it's easy to write it.
git fast-import The Migrating to Git chapter explains how you can use the git fast-import tool to manually import anything into git. I've used it to create tags with dates in the past, slightly changing the Custom Importer example in the book: 
#!/usr/bin/env ruby
#
# retag.rb
#
# Small script to create an annotated tag, specifying commiter as well as
# date, and tag comment.
#
# Based on Scott Chacon "Custom Importer" example.
#
# Arguments:
#  $1 -- tag name
#  $2 -- sha-1 revision to tag
#  $3 -- committer in the form First Last <email>
#  $4 -- date to use in the form YYYY/MM/DD_HH:MM:SS

def help
  puts "Usage: retag <tag> <sha1sum> <committer> <date> <comment>"
  puts "Creates a annotated tag with name <tag> for commit <sha1sum>, using "
  puts "given <committer>, <date> and <comment>"
  puts "The output should be piped to git fast-import"
end
def to_date(datetime)
  (date, time) = datetime.split('_')
  (year, month, day) = date.split('/')
  (hour, minute, second) = time.split(':')
  return Time.local(year, month, day, hour, minute, second).to_i
end
def generate_tag(tag, sha1hash, committer, date, message)
  puts "tag # tag "
  puts "from # sha1hash "
  puts "tagger # committer  # date  +0000"
  print "data # message.size \n# message "
end
if ARGV.length != 5
  help
  exit 1
else
  (tag, sha1sum, committer, date, message) = ARGV
  generate_tag(tag, sha1sum, committer, to_date(date), message)
end
graft points (graft means greffe in French) Because of missing svn:mergeinfo some changes appear unmerged. To fix this there are graft points: they override git idea of parents of a commit. To create a graft point, assuming 6a6d48814d0746fa4c9f6869bd8d5c3bc3af8242 is the commit you want to change, currently with a single parent 898ad49b61d4d8d5dc4072351037e2c8ade1ab68, but containing changes from commit 11cf74d4aa996ffed7c07157fe0780ec2224c73e: 
me@mymachine$ echo 6a6d48814d0746fa4c9f6869bd8d5c3bc3af8242 11cf74d4aa996ffed7c07157fe0780ec2224c73e 898ad49b61d4d8d5dc4072351037e2c8ade1ab68 >> .git/info/grafts
git filter-branch git filter-branch allows you to completely rewrite history of a git branch, changing or dropping commits while traversing the branch. As an additional benefit, this tool use graft points and make them permanent. In other words: after running git filter-branch you can remove .git/info/grafts file. I've used it to rewrite author of a given set of commits, using a hack on top of Chris Johnsen script: 
#!/bin/sh

br="HEAD"
TARG_NAME="Raphael Bossek"
TARG_EMAIL="hidden"
export TARG_NAME TARG_EMAIL
filt='
    if test "$GIT_COMMIT" = 546db1966133737930350a098057c4d563b1acdf -o \
            "$GIT_COMMIT" = 23419dde50662852cfbd2edde9468beb29a9ddcc; then
        if test -n "$TARG_EMAIL"; then
            GIT_AUTHOR_EMAIL="$TARG_EMAIL"
            export GIT_AUTHOR_EMAIL
        else
            unset GIT_AUTHOR_EMAIL
        fi
        if test -n "$TARG_NAME"; then
            GIT_AUTHOR_NAME="$TARG_NAME"
            export GIT_AUTHOR_NAME
        else
            unset GIT_AUTHOR_NAME
        fi
    fi
'
git filter-branch $force --tag-name-filter cat --env-filter "$filt" -- $br
(Script edited here; there were much more commits written by Raphael.)

Important

It's important to realize that the whole selected branch history is rewritten, so all objects id will change. You should not do this if you already published your repository.

The --tag-name-filter cat argument ensures our tags are copied during the traversal; otherwise they would be untouched, and hence not available in the new history.

Hint

Once git filter-branch completes you get a new history, as well as a new original ref to ease comparison. It is highly recommended to check the result of the rewrite before removing original. To shrink the repo after this, git clone the rewritten repo with file:// syntax -- git-filter-branch says it all.

Cleaning up the repo To recap, here's how the ACE+TAO git repo was changed after conversion:

  1. Add graft points where needed.

  2. Clean tags and branches. Using git tag -d, git branch -d and the Ruby script above it was possible to recreate tags. During this I was also able to add missing tags, and remove some SVN errors I did -- like committing in a branch created under tags/.

  3. Remove obsolete branches.

  4. Merge missing pieces. There were just two missing debian/changelog entries. I did this before git filter-branch because I did not find a way to use the tool correctly with multiple heads.

  5. Fix commit author where needed. Using the shell script above Raphael is now correctly credited for his work.

That's it. The ACE+TAO git repository for Debian packages is alive at http://git.debian.org/?p=pkg-ace/pkg-ace.git;a=summary.
[1]http://lists.alioth.debian.org/pipermail/pkg-ace-devel/2011-March/002421.html
[2]available in Debian as svn-all-fast-export

20 August 2010

Riku Voipio: (unnofficial) Bits from ARM porters

Quite a few things have happened recently in the Debian/ARM land

ARM and Canonical have generously provided us with bunch of fast armel machines. Four of these are now as buildd's, bringing the total of armel buildd's to 7. In other words, armel should no longer lag behind other architectures when building unstable packages.

One of machines, abel.debian.org has been setup as a porter box. It is faster (around 3x) than the older porterbox (agricola). All Debian Developers have access to the porterboxes.

The new buildds have allowed us to enable more suites. Thanks to Philipp Kerns work, armel builds now experimental, lenny-volatile, lenny-backports as well as unstable/non-free. Especially if you are using stable Debian, access to backports and volatile should make life happier :)

Finally, the next big thing is Hardfloat ARM port, effort being lead by Konstantinos Margaritis. This doesn't mean that the armel port is going away. Majority of ARM cpus sold are still without FPU, so the softloaf port (armel) will still have a long life ahead. Meanwhile, the armhf port will provide a more optimal platform for people with bleeding edge ARM cores (ARMv7 + vfp). Some people have been unhappy with the new proposed new port, and various alternatives have been proposed. However, armhf is currently the only solution being actively worked on.

Update: thank canonical too