My project.
This post assumes that the reader has read
my last post.
This week I had less time to work on my project than the last ones, because it was kind of the last week of the semester for some disciplines. This was unfortunate from a point of view, but it s good in another, because now I ve finished most of my disciplines and can focus more. This is also the reason why the weekly report is slightly delayed.
Most of my work this week was continuing the investigation of the segfault caused by my implementation of allocation in lines. The most interesting thing I discovered came after
a suggestion from my menthor: to use
+RTS -DS to turn on the sanity checker. I runned the code with my allocation implementation and the sanity checker and got a segfault. I runned again only with the code to free memory in lines and I got also a segfault. Then I runned without all my patches, using sweep, and I got the segfault. So it seems that there is something wrong with sweep, and now I m investigating this new segfault.
I m using the
bernouilli program from
nofib to test, running with 148 as a parameter and, naturally,
+RTS -w -DS passed to the RunTime System. The output of gdb:
Current directory is /home/marcot/trabalho/livre/ghc/nofib/imaginary/bernouilli/
GNU gdb (GDB) 7.1-debian
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/marcot/trabalho/livre/ghc/nofib/imaginary/bernouilli/Main...done.
(gdb) r 148 +RTS -w -DS
Starting program: /home/marcot/trabalho/livre/ghc/nofib/imaginary/bernouilli/Main 148 +RTS -w -DS
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0x00000000006309cb in LOOKS_LIKE_INFO_PTR_NOT_NULL (p=12297829382473034410) at includes/rts/storage/ClosureMacros.h:225
(gdb) where
#0 0x00000000006309cb in LOOKS_LIKE_INFO_PTR_NOT_NULL (p=12297829382473034410) at includes/rts/storage/ClosureMacros.h:225
#1 0x0000000000630a16 in LOOKS_LIKE_INFO_PTR (p=12297829382473034410) at includes/rts/storage/ClosureMacros.h:230
#2 0x0000000000630a4b in LOOKS_LIKE_CLOSURE_PTR (p=0x7ffff6c84062) at includes/rts/storage/ClosureMacros.h:235
#3 0x0000000000631427 in checkClosure (p=0x7ffff6853a08) at rts/sm/Sanity.c:320
#4 0x00000000006319ef in checkHeap (bd=0x7ffff68014c0) at rts/sm/Sanity.c:479
#5 0x000000000063222f in checkSanity (check_heap=rtsTrue) at rts/sm/Sanity.c:686
#6 0x000000000062dfdb in GarbageCollect (force_major_gc=rtsFalse, gc_type=0, cap=0x8d2ec0) at rts/sm/GC.c:768
#7 0x0000000000620431 in scheduleDoGC (cap=0x8d2ec0, task=0x8f5080, force_major=rtsFalse) at rts/Schedule.c:1420
#8 0x000000000061fa2c in schedule (initialCapability=0x8d2ec0, task=0x8f5080) at rts/Schedule.c:539
#9 0x0000000000620c77 in scheduleWaitThread (tso=0x7ffff6c80000, ret=0x0, cap=0x8d2ec0) at rts/Schedule.c:1902
#10 0x000000000065762b in rts_evalLazyIO (cap=0x8d2ec0, p=0x89d8b0, ret=0x0) at rts/RtsAPI.c:495
#11 0x000000000061d3db in real_main () at rts/RtsMain.c:66
#12 0x000000000061d4ca in hs_main (argc=5, argv=0x7fffffffe6d8, main_init=0x406558 <__stginit_ZCMain>, main_closure=0x89d8b0) at rts/RtsMain.c:115
#13 0x00007ffff6fbcabd in __libc_start_main (main=<value optimized out>, argc=<value optimized out>, ubp_av=<value optimized out>, init=<value optimized out>, fini=<value optimized out>, rtld_fini=<value optimized out>, stack_end=0x7fffffffe6c8) at libc-start.c:222
#14 0x0000000000403a69 in _start ()
I started the investigation with the most obvious test: to remove the call to
sweep() in
rts/sm/GC.c.
if (major_gc && oldest_gen->mark)
if (oldest_gen->compact)
compact(gct->scavenged_static_objects);
// else
// sweep(oldest_gen);
The result was the same, same segfault in the same place. So I decided to do it a little stronger and avoid the blocks getting the
BF_MARKED flag, so that they are not even marked.
if (!(bd->flags & BF_FRAGMENTED))
// bd->flags = BF_MARKED;
This one worked. The code in
sweep() is not commented, but it s irrelevant, since it ignores blocks that don t have this flag. My third try was to comment the places where the
BF_MARKED flag is read, to see each one was causing the segfault. I got a list of places to search with grep, and there weren t a lot of them.
$ grep BF_MARKED rts/sm/*.c
rts/sm/Compact.c: if (bd->flags & BF_MARKED)
rts/sm/Evac.c: if ((bd->flags & (BF_LARGE BF_MARKED BF_EVACUATED)) != 0)
rts/sm/Evac.c: if (bd->flags & BF_MARKED)
rts/sm/GCAux.c: if ((bd->flags & BF_MARKED) && is_marked((P_)q,bd))
rts/sm/GC.c: if (!(bd->flags & BF_MARKED))
rts/sm/GC.c: // time, so reset the BF_MARKED flags.
rts/sm/GC.c: // compact. (search for BF_MARKED above).
rts/sm/GC.c: bd->flags &= ~BF_MARKED;
rts/sm/GC.c: // Also at this point we set the BF_MARKED flag
rts/sm/GC.c: // BF_MARKED is always unset, except during GC
rts/sm/GC.c: bd->flags = BF_MARKED;
rts/sm/Sweep.c: if (!(bd->flags & BF_MARKED))
The first one is in
rts/sm/Compact.c, so it s not relevant to the use with
-w. The second one, in
rts/sm/Evac.c, is a bit indirect.
if ((bd->flags & (BF_LARGE BF_MARKED BF_EVACUATED)) != 0)
// pointer into to-space: just return it. It might be a pointer
// into a generation that we aren't collecting (> N), or it
// might just be a pointer into to-space. The latter doesn't
// happen often, but allowing it makes certain things a bit
// easier; e.g. scavenging an object is idempotent, so it's OK to
// have an object on the mutable list multiple times.
if (bd->flags & BF_EVACUATED)
// We aren't copying this object, so we have to check
// whether it is already in the target generation. (this is
// the write barrier).
if (bd->gen < gct->evac_gen)
gct->failed_to_evac = rtsTrue;
TICK_GC_FAILED_PROMOTION();
return;
/* evacuate large objects by re-linking them onto a different list.
*/
if (bd->flags & BF_LARGE)
info = get_itbl(q);
if (info->type == TSO &&
((StgTSO *)q)->what_next == ThreadRelocated)
q = (StgClosure *)((StgTSO *)q)->_link;
*p = q;
goto loop;
evacuate_large((P_)q);
return;
/* If the object is in a gen that we're compacting, then we
* need to use an alternative evacuate procedure.
*/
if (!is_marked((P_)q,bd))
mark((P_)q,bd);
push_mark_stack((P_)q);
return;
The first if, in line 466, is executed if any of the three flags is present:
BF_LARGE,
BF_MARKED or
BF_EVACUATED. The second if, in line 474, checks for
BF_EVACUATED, and returns. The third if, in line 487, checks for
BF_LARGE and returns. The code in lines 502-505 is only executed if
BF_MARKED is present, and not the other ones. I tried commenting this code, and got an assertion fail in the user code, so I think this is not a good path to follow.
The second occurence of
BF_MARKED is in the same file.
if (bd->flags & BF_MARKED)
// must call evacuate() to mark this closure if evac==rtsTrue
*q = (StgClosure *)p;
if (evac) evacuate(q);
unchain_thunk_selectors(prev_thunk_selector, (StgClosure *)p);
return;
Commenting it, with or without the call to
sweep() commented, causes the same segfault. So, I m tending to think this part of the code is unrelated to the issue.
The third occurence is in
rts/sm/GCAux.c. I tried commenting it with all the four combinations of the two others commented and not commented, and all resulted in the segfault in the same place.
There s another place to check in
rts/sm/GC.c. Again, commenting it made no difference. The last one is part of
sweep(), so it s avoided anyway when the call to this function is commented.
I hate keeping track of money. Bank accounts, credit cards,
investing it's such a hassle. Hell, sometimes I even hate the fact
that money exists and needs to be dealt with in the first place. But I
admit that it's an easier system than bartering for everything.
In college I pretty much ignored most things financial. I rarely had
more money than I needed, and put little financial planning into
deciding what I could and could not spend money on besides, "I'm going
to make X dollars this summer," and, "think twice." But now that I
have a steady paycheck, it seems like a good idea to know where all my
money goes, so I can make better decisions about how I'm spending (and
saving) it.
But I've kept procrastinating starting to do so, because GUI programs
like Gnucash and
HomeBank seem like such a hassle. Their first
screen is dauntingly complex, and if you don't know much about
accounting it's scary and difficult to be asked to set up a big set of
accounts when first starting to use the program, without having any
prior experience with what you personally would find useful to keep
track of. Entering data through menus and dialogs is tedious and slow.
hledger (or
ledger, which came first) had
been appearing on my radar recently, not least because
Iron Blogger uses it). hledger
changed my opinion of accounting software. In about 15 minutes, using
only the sample transactions from
the manual, I was able to enter all my
assets and liabilities bank accounts, credit card, student loans,
money I've borrowed from people and never paid back though I said I
would, etc. And then, I could run 'hledger balance' and it would tell me
what sorts of things I had spent money on in the past few days, as far
back as I could bother looking up actual transactions for, rather than
entering a single transaction with a balance forward. Duuude, awesome!
I'm a geek. I like statistics and data about my life, as long as it's
not a huge pain to collect the data in the first place. More data means
more on which to base decisions, decisions which will then be freer from
the bias of what I find memorable enough to remember having done.
Here's what I like best about ledger:
Since a few months, I have been playing around with 
At 
So, flipping through the otherwise good 
Sometimes Wikipedia shows it even has a somewhat humorous side. My finding it funny may be a product of late night learning sessions and semi-random clicking-sprees, but still, worthy of a chuckle for those with the right (wrong?) inclinations.
Folks at 
Funny how life brings those A-HA moments in the least probable of circumstances. Take my career's development, for instance. I'm currently taking a training to become a government bureaucrat. During this training, each student goes through a thorough skill assessment to help the trainer select an appropriate training place in some government office. Asides from covering the obvious aspects of formal education and employment history, we also reviewed achievements and accumulated skills. My A-HA moment came as a result of our trainer asking me to completely rewrite my CV to match some known template. Additionally, she requested that I emphasize my technology background more. I countered by pointing out that the word "technology" tends to mean "engineer" to the average employer and yet I've have always been in Product Management or Business Development, which are more Sales-oriented roles than anything else, hence why I emphasize the Sales aspect and deflate the technology aspect in my CV. Still, while she conceded that difference in emphasis, she insisted that having worked in technology probably influenced my skills or my preferred workflow in one way or another. At the moment, I simply could not think of how it might have.
Boy, was I wrong! Where do I begin?
First of all, it dawned onto me that I simply cannot claim to know Microsoft Office anymore. Until recently, my Open Office skills were easily applicable to Microsoft Office, because Open Office borrowed a lot of concepts from its Microsoft counterpart. However, following the recent redesign of Microsoft Office, I found that I cannot navigate my way around Microsoft Word's menus anymore. While this new user interface paradigm indeed removes a lot of clutter, it also hides too many features in less than obvious places, which resulted in me concluding that I simply have to downgrade my Microsoft Office skills to medium. Given the progressive conversion of several Finnish agencies and ministries to Open Office, I'm not in such an uncomfortable position but, then again, other agencies and ministries fiercely cling on to their Microsoft licenses and have recently upgraded them. As such, should my practical training take place in one of those offices, I would essentially be unable to perform at my job. Redeeming factor: a friend who only knows Microsoft products faced similar frustrations last year when she got back from her summer vacations, after she realized that their network administrator had upgraded her workstation. It took her the whole autumn before she felt comfortable using Word again.
Another aspect of working in Free Software that influences my workflow: telecommuting and teleworking. As anyone working on Free Software projects knows, teams tend to be distributed around the globe, which means that there's always someone somewhere pushing a commit or answering bug reports, at any given time of the day. Simultaneously, work quickly becomes location-independent and flexible schedules are the norm; whatever and wherever works for a given developer, as long as the work gets done. Without anyone really noticing, this work methodology has permeated the whole technology ecosystem, even at fortune-500 companies. Employees come in and out of the office at whichever time suits them, while others choose to work from home and only show up whenever face-to-face meetings are called. Others even adopt a nomad lifestyle, constantly roaming the globe for adventures and connecting to the office network via VPN, from the comfort of their hotel room or from a friend's couch, on the other side of the globe.
In my case, having twice worked for Estonian companies while living in Finland, it meant taking the ferry twice a week to visit the office. This brought in more benefits than one might initially think: first of all, the quick walk between the metro station and the harbor in Helsinki meant that I arrived on the ferry with blood pumping adrenaline and fully alert. Being on the ferry gave me 2 hours of quiet time to grab my first coffee and plan my day. Getting out of the ferry in Tallinn meant another quick walk, this time between the harbor and the tramway. By the time I arrived at the office, I had exercised twice and planned all my workday. It's probably the most productive that I've ever been in my whole career. Additionally, I was frequently on the road, meeting customers and following on sales leads, which meant that I got to close many deals using my laptop in my hotel room. As our CEO used to joke, "I have no idea where in the world Martin- ric is today but, just as long as the purchase orders keep on pouring in, it's all the same to me."
As a result of this reflexion, I had to explain to our trainer why I am extremely well-suited to government jobs that require a lot of traveling and where Free Software is used on the desktop and, vice-versa, extremely unsuited to back-office jobs where whatever Microsoft products of the day are the norm. I've had to put particular emphasis on what Open Office is all about, because many homeworks were supposed to be submitted in Microsoft Word format. While Writer indeed offers the option to import and export Microsoft formats, it doesn't come with any WYSIWYG guarantee, which is why I took on the habit of sending her PDF documents. Sadly, this did not always work out so well, especially in cases where the intention was to forward selected parts of a document to third-parties.
Conclusion: working in the Free Software industry, even in non-engineering roles, indeed has a strong influence on someone's choice of methodology, tools and workflow. In some cases, it can even disqualify someone from making certain career choices.
Who would have thought?
The GNOME Foundation released their 
