Manipur Videos
Warning: The text might be mature and will have references to violence so if there are kids or you are sensitive, please excuse.
Few days back, saw the videos and I cannot share the rage, shame and many conflicting emotions that were going through me. I almost didn t want to share but couldn t stop myself. The woman in the video were being palmed, fingered, nude, later reportedly raped and murdered. And there have been more than a few cases. The next day saw another video that showed beheaded heads, and Kukis being killed just next to their houses. I couldn t imagine what those people must be feeling as the CM has been making partisan statements against them. One of the husbands of the Kuki women who had been paraded, fondled is an Army Officer in the Indian Army. The Meiteis even tried to burn his home but the Army intervened and didn t let it get burnt.
The CM s own statement as shared before tells his inability to bring the situation out of crisis. In fact, his statement was dumb stating that the Internet shutdown was because there were more than 100 such cases. And it s spreading to the nearby Northeast regions. Now Mizoram, the nearest neighbor is going through similar things where the Meitis are not dominant. The Mizos have told the Meitis to get out.
To date, the PM has chosen not to visit Manipur. He just made a small 1 minute statement about it saying how the women have shamed India, an approximation of what he said.While it s actually not the women but the men who have shamed India. The Wire has been talking to both the Meitis, the Kukis, the Nagas. A Kuki women sort of bared all. She is right on many counts. The GOI while wanting to paint the Kukis in a negative light have forgotten what has been happening in its own state, especially its own youth as well as in other states while also ignoring the larger geopolitics and business around it. Taliban has been cracking as even they couldn t see young boys, women becoming drug users. I had read somewhere that 1 in 4 or 1 in 5 young person in Afghanistan is now in its grip. So no wonder,the Taliban is trying to eradicate and shutdown drug use among it s own youth.
Circling back to Manipur, I was under the wrong impression that the Internet shutdown is now over. After those videos became viral as well as the others I mentioned, again the orders have been given and there is shutdown. It is not fully shut but now only Govt. offices have it. so nobody can share a video that goes against any State or Central Govt. narrative A real sad state of affairs
Update: There is conditional reopening whatever that means
When I saw the videos, the first thing is I felt was being powerless, powerless to do anything about it. The second was if I do not write about it, amplify it and don t let others know about it then what s the use of being able to blog
Mental Health, Binging on various Webseries
Both the videos shocked me and I couldn t sleep that night or the night after. it. Even after doing work and all, they would come in unobtrusively in my nightmares While I felt a bit foolish, I felt it would be nice to binge on some webseries. Little I was to know that both Northshore and Alaska Daily would have stories similar to what is happening here While the story in Alaska Daily is fictional it resembles very closely to a real newspaper called Anchorage Daily news. Even there the Intuit women , one of the marginalized communities in Alaska. The only difference I can see between GOI and the Alaskan Government is that the Alaskan Government was much subtle in doing the same things. There are some differences though. First, the State is and was responsive to the local press and apart from one close call to one of its reporters, most reporters do not have to think about their own life in peril. Here, the press cannot look after either their livelihood or their life. It was a juvenile kid who actually shot the video, uploaded and made it viral. One needs to just remember the case details of Siddique Kappan. Just for sharing the news and the video he was arrested. Bail was denied to him time and time again citing that the Police were investigating . Only after 2 years and 3 months he got bail and that too because none of the charges that the Police had they were able to show any prima facie evidence.
One of the better interviews though was of Vrinda Grover. For those who don t know her, her Wikipedia page does tell a bit about her although it is woefully incomplete. For example, most recently she had relentlessly pursued the unconstitutional Internet Shutdown that happened in Kashmir for 5 months. Just like in Manipur, the shutdown was there to bury crimes either committed or being facilitated by the State. For the issues of livelihood, one can take the cases of Bipin Yadav and Rashid Hussain. Both were fired by their employer Dainik Bhaskar because they questioned the BJP MP Smriti Irani what she has done for the state. The problems for Dainik Bhaskar or for any other mainstream media is most of them rely on Government advertisements. Private investment in India has fallen to record lows mostly due to the policies made by the Centre. If any entity or sector grows a bit then either Adani or Ambani will one way or the other take it. So, for most first and second generation entrepreneurs it doesn t make sense to grow and then finally sell it to one of these corporates at a loss GOI on Adani, Ambani side of any deal. The MSME sector that is and used to be the second highest employer hasn t been able to recover from the shocks of demonetization, GST and then the pandemic. Each resulting in more and more closures and shutdowns. Most of the joblessness has gone up tremendously in North India which the Government tries to deny.
The most interesting points in all those above examples is within a month or less, whatever the media reports gets scrubbed. Even the firing of the journos that was covered by some of the mainstream media isn t there anymore. I have to use secondary sources instead of primary sources. One can think of the chilling effects on reportage due to the above. The sad fact is even with all the money in the world the PM is unable to come to the Parliament to face questions.
The above poster/question is by Surya Pratap Singh, a retired IAS officer. He asks why the PM is unable to answer in either of the houses. As shared before, the Govt. wants very limited discussion. Even yesterday, the Lok Sabha TV just showed the BJP MP s making statements but silent or mic was off during whatever questions or statements made by the opposition. If this isn t mockery of Indian democracy then I don t know what is
Even the media landscape has been altered substantially within the last few years. Both Adani and Ambani have distributed the media pie between themselves. One of the last bastions of the free press, NDTV was bought by Adani in a hostile takeover. Both Ambani and Adani are close to this Goverment. In fact, there is no sector in which one or the other is not present. Media houses like Newsclick, The Wire etc. that are a fraction of mainstream press are where most of the youth have been going to get their news as they are not partisan. Although even there, GOI has time and again interfered. The Wire has had too many 504 Gateway timeouts in the recent months and they had been forced to move most of their journalism from online to video, rather Youtube in order to escape both the censoring and the timeouts as shared above. In such a hostile environment, how both the organizations are somehow able to survive is a miracle. Most local reportage is also going to YouTube as that s the best way for them to not get into Govt. censors. Not an ideal situation, but that s the way it is.
The difference between Indian and Israeli media can be seen through this
The above is a Screenshot shared by how the Israeli media has reacted to the Israeli Government s Knesset over the judicial overhaul . Here, the press itself erodes its own by giving into the Government day and night
Binging on Webseries
Saw Northshore, Three Pines, Alaska Daily and Doogie Kamealoha M.D. which is based on Doogie Howser M.D. Of the four, enjoyed Doogie Kamealoha M.D. the most but then it might be because it s a copy of Doogie Howser, just updated to the new millenia and there are some good childhood memories associated with that series. The others are also good. I tried to not see European stuff as most of them are twisted and didn t want that space.
EU Digital Operational Resilience Act and impact on FOSS
Few days ago, apparently the EU shared the above Act. One can read about it more here. This would have more impact on FOSS as most development of various FOSS distributions happens in EU. Fair bit of Debian s own development happens in Germany and France. While there have been calls to make things more clearer, especially for FOSS given that most developers do foss development either on side or as a hobby while their day job is and would be different. The part about consumer electronics and FOSS is a tricky one as updates can screw up your systems. Microsoft has had a huge history of devices not working after an update or upgrade. And this is not limited to Windows as they would like to believe. Even apple seems to be having its share of issues time and time again. One would have hoped that these companies that make billions of dollars from their hardware and software sales would be doing more testing and Q&A and be more aware about security issues. FOSS, on the other hand while being more responsive doesn t make as much money vis-a-vis the competitors. Let s take the most concrete example. The most successful mobile phone having FOSS is Purism. But it s phone, it has priced itself out of the market. A huge part of that is to do with both economies of scale and trying to get an infrastructure and skills in the States where none or minimally exists. Compared that to say Pinepro that is manufactured in Hong Kong and is priced 1/3rd of the same.
For most people it is simply not affordable in these times. Add to that the complexity of these modern cellphones make it harder, not easier for most people to be vigilant and update the phone at all times. Maybe we need more dumphones such as Light and Punkt but then can those be remotely hacked or not, there doesn t seem to be any answers on that one. I haven t even seen anybody even ask those questions. They may have their own chicken and egg issues.
For people like me who have lost hearing, while I can navigate smartphones for now but as I become old I don t see anything that would help me. For many an elderly population, both hearing and seeing are the first to fade. There doesn t seem to be any solutions targeted for them even though they are 5-10% of any population at the very least. Probably more so in Europe and the U.S. as well as Japan and China. All of them are clearly under-served markets but dunno a solution for them. At least to me that s an open question.
RISC -V Motherboard, SBC
While I didn t want to, a part of me is hyped about this motherboard. This would probably be launched somewhere in November. There are obvious issues in this, the first being unlike regular motherboards you wouldn t be upgrade as you would do.You can t upgrade your memory, can t upgrade the CPU (although new versions of instructions could be uploaded, similar to BIOS updates) but as the hardware is integrated (the quad-core SiFive Performance P550 core complex) it would really depend. If the final pricing is around INR 4-5k then it may be able to sell handsomely provided there are people to push and provide support around it. A 500 GB or 1 TB SSD coupled with it and a cheap display unit and you could use it anywhere although as the name says it s more for tinkering as the name suggests.
Another board that could perhaps be of more immediate use would be the beagleboard. They launched the same couple of days back and called it Beagle V-Ahead. Again, costs are going to be a concern. Just a year before the pandemic the Beagleboard Black (BB) used to cost in the sub 4k range, today it costs 8k+ for the end user, more than twice the price. How much Brexit is to be blamed for this and how much the Indian customs we would never know. The RS Group that is behind that shop is head-quartered in the UK.
As said before, we do not know the price of either board as it probably will take few months for v-ahead to worm its way in the Indian market, maybe another 6 months or so. Even so, with the limited info. on both the boards, I am tilting more towards the other HiFive one. We should come to know about the boards say in 3-5 months of time.
CHIPS Act
I had shared about the Chips Act a few times here as well as on SM. Twoarticles do tell how the CHIPS Act 2023 is more of a political tool, an industrial defence policy rather than just business as most people tend to think.
Cancelation of Books, Books Burning etc.
Almost 2400 years ago, Plato released his work called Plato s Republic and one of the seminal works within it is perhaps one of the most famous works was the Allegory of the Cave. That is used again and again in a myriad ways, mostly in science-fiction though and mostly to do with utopian, dystopian movies, webseries etc. I did share how books are being canceled in the States, also a bit here. But the most damning thing has happened throughout history, huge quantities of books burned almost all for politics But part of it has been neglect as well as this time article shares. What we have lost and continue to lose is just priceless. Every book has a grain of truth in it, some more, some less but equally enjoyable.
Most harmful is the neglect towards books and is more true today than any other time in history. Kids today have a wide variety of tools to keep themselves happy or occupied, from anime, VR, gaming the list goes on and on. In that scenario, how the humble books can compete. People think of Kindle but most e-readers like Kindle are nothing but obsolescence by design. I have tried out Kindle a few times but find it a bit on the flimsy side. Books are much better IMHO or call me old-school. While there are many advantages, one of the things that I like about books is that you can easily put yourself in either the protagonist or the antagonist or somewhere in the middle and think of the possible scenarios wherever you are in a particular book. I could go on but it will be a blog post or two in itself.
Till later. Happy Reading.
Update:Manipur
Extremely horrifying visuals, articles and statements continue to emanate from Manipur. Today, 19th July 2023, just couple of hours back, a video surfaced showing two Kuki women were shown as stripped, naked and Meitei men touching their private parts. Later on, we came to know that this was in response of a disinformation news spread by the Meitis of few women being raped although no documentary evidence of the same surfaced, no names nothing. While I don t want to share the video I will however share the statement shared by the Kuki-Zo tribal community on that. The print gives a bit more context to what has been going on.
Update, Few hours later : The Print also shared more of a context about six days ago. The reason we saw the video now was that for the last 2.5 months Manipur was in Internet shutdown so those videos got uploaded now. There was huge backlash from the Twitter community and GOI ordered the Manipur Police to issue this Press Release yesterday night or just few hours before with yesterday s time-stamp.
IndianExpress shared an article that does state that while an FIR had been registered immediately no arrests so far and this is when you can see the faces of all the accused. Not one of them tried to hide their face behind a mask or something. So, if the police wanted, they could have easily identified who they are. They know which community the accused belong to, they even know from where they came. If they wanted to, they could have easily used mobile data and triangulation to find the accused and their helpers. So, it does seem to be attempt to whitewash and protect a certain community while letting it prey on the other.
Another news that did come in, is because of the furious reaction on Twitter, Youtube has constantly been taking down the video as some people are getting a sort of high more so from the majoritarian community and making lewd remarks. Twitter has been somewhat quick when people are making lewd remarks against the two girl/women. Quite a bit of the above seems like a cover-up.
Lastly, apparently GOI has agreed to having a conversation about it in Lok Sabha but without any voting or passing any resolutions as of right now. Would update as an when things change.
Update: Smriti Irani, the Child and Development Minister gave the weakest statement possible
As can be noticed, she said sexual assault rather than rape. The women were under police custody for safety when they were whisked away by the mob. No mention of that. She spoke to the Chief Minister who has been publicly known as one of the provocateurs or instigators for the whole thing. The CM had publicly called the Kukus and Nagas as foreigners although both of them claim to be residing for thousands of years and they apparently have documentary evidence of the same . Also not clear who is doing the condemning here. No word of support for the women, no offer of intervention, why is she the Minister of Child and Women Development (CDW) if she can t use harsh words or give support to the women who have gone and going through horrific things
Update : CM Biren Singh s Statement after the video surfaced
This tweet is contradictory to the statements made by Mr. Singh couple of months ago. At that point in time, Mr. Singh had said that NIA, State Intelligence Departments etc. were giving him minute to minute report on the ground station. The Police itself has suo-moto (on its own) powers to investigate and apprehend criminals for any crime. In fact, the Police can call for questioning of anybody in any relation to any crime and question them for upto 48 hours before charging them. In fact, many cases have been lodged where innocent persons have been framed or they have served much more in the jail than the crime they are alleged to have been committed. For e.g. just a few days before there was a media report of a boy who has been in jail for 3 years. His alleged crime, stealing mere INR 200/- to feed himself. Court doesn t have time to listen to him yet. And there are millions like him.
The quint eloquently shares the tale where it tells how both the State and the Centre have been explicitly complicit in the incidents ravaging Manipur. In fact, what has been shared in the article has been very true as far as greed for land is concerned. Just couple of weeks back there have been a ton of floods emanating from Uttarakhand and others. Just before the flooding began, what was the CM doing can be seen here.
Apart from the newspapers I have shared and the online resources, most of the mainstream media has been silent on the above. In fact, they have been silent on the Manipur issue until the said video didn t come into limelight. Just now, in Lok Sabha everybody is present except the Prime Minister and the Home Minister. The PM did say that the law will take its own course, but that s about it. Again no support for the women concerned.
Update: CJI (Chief Justice of India) has taken suo-moto cognizance and has warned both the State and Centre to move quickly otherwise they will take the matter in their own hand.
Update: Within 2 hours of the CJI taking suo-moto cognizance, they have arrested one of the main accused Heera Das
The above tells you why the ban on Internet was put in the first place. They wanted to cover it all up. Of all the celebs, only one could find a bit of spine, a bit of backbone to speak about it, all the rest mum
Just imagine, one of the women is around my age while the young one could have been a daughter if I had married on time or a younger sister for sure. If ever I came face to face with them, I just wouldn t be able to look them in the eye. Even their whole whataboutery is built on sham. From their view Kukis are from Burma or Burmese descent. All of which could be easily proved by DNA of all. But let s leave that for a sec. Let s take their own argument that they are Burmese. Their idea of Akhand Bharat stretches all the way to Burma (now called Myanmar). They want all the land but no idea with what to do with the citizens living on it.
Even after the video, the whataboutery isn t stopping, that shows how much hatred is there. And not knowing that they too will be victim of the same venom one or the other day Update: Opposition was told there would be a debate on Manipur. The whole day went by, no debate. That s the shamelessness of this Govt. Update 20th July 19:25 Center may act or not act against the perpetrators but they will act against Twitter who showed the crime. Talk about shooting the messenger
We are now in the last stage. In 2014, we were at 6
It s been 2 years since my last blog post about storage trends [1].
Minimum Storage <=2TB
In 2021 I stated that as MSY had 2TB disks for $72 and 2TB SSD for $245 it was barely worth considering a 2TB disk and anything less than 2TB wasn t worth considering. Now for 2TB storage from MSY NVMe starts at $129, SATA SSD starts at $143, and hard disks start at $75. I guess that NVMe is slightly cheaper due to some combination of economies of scale for manufacture/sales and having less postage costs. It really doesn t make sense to consider hard disks for storing 2TB or less.
For storage for a small system (PC or laptop) the cheapest storage device is $19 for a 128G SATA SSD. But it wouldn t make sense to buy that when you can get a 256G SATA SSD for $22 or a 240G NVMe device for $23, saving $3 on storage wouldn t make any sense. For 512G of storage the prices are $32 for NVMe and $33 for SATA SSD. For 1TB of storage the prices start at $68 for SATA SSD and $74 for NVMe. Probably for the vast majority of home users 1TB of SATA SSD or NVMe is the minimum storage capacity to consider, the $50 price difference isn t much when considering the entire price of a PC or laptop and anything less than 1TB will run out quickly with modern use.
Larger Storage 4TB+
The price for 4TB of storage from MSY is NVMe starting at $349, SATA SSD starting at $369, and hard disks starting at $115. If you need 4TB of RAID-1 storage then it might be worth saving $470 and getting hard drives for a home user. For business use it wouldn t make sense. Some laptops have two NVMe sockets so 8TB of storage (or 4TB of RAID-1) in a laptop would be interesting.
For 8TB of storage the MSY prices are SATA SSD for $739 and hard drives starting at $179. Probably hard drives are the best choice for most situations where there is a need to store 8TB or more of data. But the prices are low enough to make 8TB SSD something that can be considered for home use, it doesn t seem that long ago that the 4TB hard drives I bought for my home server were almost that expensive.
Big Storage
MSY doesn t have 8TB NVMe, such devices are on eBay for $1700 for regular M.2 NVMe and just under $1000 for U.2 (server hot-swap devices). So if you need more than 8TB of NVMe storage then probably buying a server with U.2 built in is the correct solution.
For home users who need more than 8TB of storage hard drives are a good solution. One issue is that the more affordable and larger drives use Shingled Magnetic Recording (SMR) which has some different performance characteristics for certain workloads. Apparently SMR performs badly for anything other than large file storage.
Why MSY?
I primarily used MSY prices for this post because they are a reliable local store that has a list of prices that is easy to read. For everything in this post I can get better prices by using eBay, the StaticIce.com.au price comparison site [2], and the computing section of the OzBargain site that gamifies finding good prices [3]. But a good shopping strategy nowadays is to compare prices in a store to determine what items are in your price range and then shop around for price on the item you want. Checking all the different bargain sites for all these items would take much more time than I want to spend writing a blog post!
Conclusion
Hard drives don t make sense for the vast majority of systems. Not for laptops, not for typical desktop PCs, and not for small business servers (say 8TB or less of RAID storage). Hard drives only make sense for dozens or hundreds of TB of storage and even then finding out how to deal with SMR issues is going to increase the pain of deployment. Maybe using a combination of SSD and hard drives to deal with the SMR issues is going to be a competitive advantage for NAS vendors in future.
NVMe looks like it s on the way to being cheaper than SATA SSD. There is likely to be a good market for systems with NVMe as the only internal storage option.
The long term trend of systems without DVD drives and with maybe 2.5 SATA devices but no 3.5 SATA devices seems to lead to the GPU being the major part that needs to fit into a PC case that determines the overall size. Maybe there will be a new trend of GPUs connected to riser cards so they can be parallel to the motherboard for compact PCs.
For business desktop systems (IE low powered graphics hardware as it s not for gaming) I expect that the trend will be towards NUC type devices which are already based around the M.2 as a storage device size.
In considering storage trends for the consumer side I m looking at the current prices from MSY (where I usually buy computer parts). I know that other stores will have slightly different prices but they should be very similar as they all have low margins and wholesale prices are the main factor.
Small Hard Drives Aren t Viable
The cheapest hard drive that MSY sells is $68 for 500G of storage. The cheapest SSD is $49 for 120G and the second cheapest is $59 for 240G. SSD is cheaper at the low end and significantly faster. If someone needed about 500G of storage there s a 480G SSD for $97 which costs $29 more than a hard drive. With a modern PC if you have no hard drives you will notice that it s quieter. For anyone who s buying a new PC spending an extra $29 is definitely worthwhile for the performance, low power use, and silence.
The cheapest 1TB disk is $69 and the cheapest 1TB SSD is $159. Saving $90 on the cost of a new PC probably isn t worth while.
For 2TB of storage the cheapest options are Samsung NVMe for $339, Crucial SSD for $335, or a hard drive for $95. Some people would choose to save $244 by getting a hard drive instead of NVMe, but if you are getting a whole system then allocating $244 to NVMe instead of a faster CPU would probably give more benefits overall.
Computer stores typically have small margins and computer parts tend to quickly either become cheaper or be obsoleted by better parts. So stores don t want to stock parts unless they will sell quickly. Disks smaller than 2TB probably aren t going to be profitable for stores for very long. The trend of SSD and NVMe becoming cheaper is going to make 2TB disks non-viable in the near future.
NVMe vs SSD
M.2 NVMe devices are at comparable prices to SATA SSDs. For some combinations of quality and capacity NVMe is about 50% more expensive and for some it s slightly cheaper (EG Intel 1TB NVMe being cheaper than Samsung EVO 1TB SSD). Last time I checked about half the motherboards on sale had a single M.2 socket so for a new workstation that doesn t need more than 2TB of storage (the largest NVMe that MSY sells) it wouldn t make sense to use anything other than NVMe.
The benefit of NVMe is NOT throughput (even though NVMe devices can often sustain over 4GB/s), it s low latency. Workstations can t properly take advantage of this because RAM is so cheap ($198 for 32G of DDR4) that compiles etc mostly come from cache and because most filesystem writes on workstations aren t synchronous. For servers a large portion of writes are synchronous, for example a mail server can t acknowledge receiving mail until it knows that it s really on disk, so there s a lot of small writes that block server processes and the low latency of NVMe really improves performance. If you are doing a big compile on a workstation (the most common workstation task that uses a lot of disk IO) then the writes aren t synchronised to disk and if the system crashes you will just do all the compilation again. While NVMe doesn t give a lot of benefit over SSD for workstation use (I ve uses laptops with SSD and NVMe and not noticed a great difference) of course I still want better performance. ;)
Last time I checked I couldn t easily buy a PCIe card that supported 2*NVMe cards, I m sure they are available somewhere but it would take longer to get and probably cost significantly more than twice as much. That means a RAID-1 of NVMe takes 2 PCIe slots if you don t have an M.2 socket on the motherboard. This was OK when I installed 2*NVMe devices on a server that had 18 disks and lots of spare PCIe slots. But for some systems PCIe slots are an issue.
My home server has all PCIe slots used by a video card and Ethernet cards and the BIOS probably won t support booting from NVMe. It s a Dell server so I can t just replace the motherboard with one that has more PCIe slots and M.2 on the motherboard. As it s running nicely and doesn t need replacing any time soon I won t be using NVMe for home server stuff.
Small Servers
Most servers that I am responsible for have less than 2TB of storage. For my clients I now only recommend SSD storage for small servers and am recommending SSD for replacing any failed disks.
My home server has 2*500G SSDs in a BTRFS RAID-1 for the root filesystem, and 3*4TB disks in a BTRFS RAID-1 for storing big files. I bought the SSDs when 500G SSDs were about $250 each and bought 2*4TB disks when they were about $350 each. Currently that server has about 3.3TB of space used and I could probably get it down to about 2.5TB if I deleted things I don t really need. If I was getting storage for that server now I d use 2*2TB SSDs and 3*1TB hard drives for the stuff that doesn t fit on SSDs (I have some spare 1TB disks that came with servers). If I didn t have spare hard drives I d get 3*2TB SSDs for that sort of server which would give 3TB of BTRFS RAID-1 storage.
Last time I checked Dell servers had a card for supporting M.2 as an optional extra so Dells probably won t boot from NVMe without extra expense.
Ars Technica has an informative article about WD selling SMR disks as NAS disks [1]. The Shingled Magnetic Recording technology allows greater storage density on a platter which leads to either larger capacity or cheaper disks but at the cost of lower write performance and apparently extremely bad latency in some situations. NAS disks are supposed to be low latency as the expectation is that they will be used in a RAID array and kicked out of the array if they have problems. There are reports of ZFS kicking SMR disks from RAID sets. I think this will end the use of hard drives for small servers. For a server you don t want to deal with this sort of thing, by definition when a server goes down multiple people will stop work (small server implies no clustering). Spending extra to get SSDs just to avoid the risk of unexpected SMR would be a good plan.
Medium Servers
The largest SSD and NVMe devices that are readily available are 2TB but 10TB disks are commodity items, there are reports of 20TB hard drives being available but I can t find anyone in Australia selling them.
If you need to store dozens or hundreds of terabytes than hard drives have to be part of the mix at this time. There s no technical reason why SSDs larger than 10TB can t be made (the 2.5 SATA form factor has more than 5* the volume of a 2TB M.2 card) and it s likely that someone sells them outside the channels I buy from, but probably at a price higher than what my clients are willing to pay. If you want 100TB of affordable storage then a mid range server like the Dell PowerEdge T640 which can have up to 18*3.5 disks is good. One of my clients has a PowerEdge T630 with 18*3.5 disks in the 8TB-10TB range (we replace failed disks with the largest new commodity disks available, it used to have 6TB disks). ZFS version 0.8 introduced a Special VDEV Class which stores metadata and possibly small data blocks on faster media. So you could have some RAID-Z groups on hard drives for large storage and the metadata on a RAID-1 on NVMe for fast performance. For medium size arrays on hard drives having a find / operation take hours is not uncommon, for large arrays having it take days isn t that uncommon. So far it seems that ZFS is the only filesystem to have taken the obvious step of storing metadata on SSD/NVMe while bulk data is on cheap large disks.
One problem with large arrays is that the vibration of disks can affect the performance and reliability of nearby disks. The ZFS server I run with 18 disks was originally setup with disks from smaller servers that never had ZFS checksum errors, but when disks from 2 small servers were put in one medium size server they started getting checksum errors presumably due to vibration. This alone is a sufficient reason for paying a premium for SSD storage.
Currently the cost of 2TB of SSD or NVMe is between the prices of 6TB and 8TB hard drives, and the ratio of price/capacity for SSD and NVMe is improving dramatically while the increase in hard drive capacity is slow. 4TB SSDs are available for $895 compared to a 10TB hard drive for $549, so it s 4* more expensive on a price per TB. This is probably good for Windows systems, but for Linux systems where ZFS and special VDEVs is an option it s probably not worth considering. Most Linux user cases where 4TB SSDs would work well would be better served by smaller NVMe and 10TB disks running ZFS. I don t think that 4TB SSDs are at all popular at the moment (MSY doesn t stock them), but prices will come down and they will become common soon enough. Probably by the end of the year SSDs will halve in price and no hard drives less than 4TB will be viable.
For rack mounted servers 2.5 disks have been popular for a long time. It s common for vendors to offer 2 versions of a rack mount server for 2.5 and 3.5 disks where the 2.5 version takes twice as many disks. If the issue is total storage in a server 4TB SSDs can give the same capacity as 8TB HDDs.
SMR vs Regular Hard Drives
Rumour has it that you can buy 20TB SMR disks, I haven t been able to find a reference to anyone who s selling them in Australia (please comment if you know who sells them and especially if you know the price). I expect that the ZFS developers will soon develop a work-around to solve the problems with SMR disks. Then arrays of 20TB SMR disks with NVMe for special VDEVs will be an interesting possibility for storage. I expect that SMR disks will be the majority of the hard drive market by 2023 if hard drives are still on the market. SSDs will be large enough and cheap enough that only SMR disks will offer enough capacity to be worth using.
I think that it is a possibility that hard drives won t be manufactured in a few years. The volume of a 3.5 disk is significantly greater than that of 10 M.2 devices so current technology obviously allows 20TB of NVMe or SSD storage in the space of a 3.5 disk. If the price of 16TB NVMe and SSD devices comes down enough (to perhaps 3* the price of a 20TB hard drive) almost no-one would want the hard drive and it wouldn t be viable to manufacture them.
It s not impossible that in a few years time 3D XPoint and similar fast NVM technologies occupy the first level of storage (the ZFS special VDEV , OS swap device, log device for database servers, etc) and NVMe occupies the level for bulk storage with no space left in the market for spinning media.
Computer Cases
For servers I expect that models supporting 3.5 storage devices will disappear. A 1RU server with 8*2.5 storage devices or a 2RU server with 16*2.5 storage devices will probably be of use to more people than a 1RU server with 4*3.5 or a 2RU server with 8*3.5 .
My first IBM PC compatible system had a 5.25 hard drive, a 5.25 floppy drive, and a 3.5 floppy drive in 1988. My current PC is almost a similar size and has a DVD drive (that I almost never use) 5 other 5.25 drive bays that have never been used, and 5*3.5 drive bays that I have never used (I have only used 2.5 SSDs). It would make more sense to have PC cases designed around 2.5 and maybe 3.5 drives with no more than one 5.25 drive bay.
The Intel NUC SFF PCs are going in the right direction. Many of them only have a single storage device but some of them have 2*M.2 sockets allowing RAID-1 of NVMe and some of them support ECC RAM so they could be used as small servers.
A USB DVD drive costs $36, it doesn t make sense to have every PC designed around the size of an internal DVD drive that will probably only be used to install the OS when a $36 USB DVD drive can be used for every PC you own.
The only reason I don t have a NUC for my personal workstation is that I get my workstations from e-waste. If I was going to pay for a PC then a NUC is the sort of thing I d pay to have on my desk.
Graph of subscribers and moderators over time in /r/NoSleep. The image is taken from our 2016 CHI paper.
Last year at CHI 2016, my research group published a qualitative study examining the effects of a large influx of newcomers to the /r/nosleep online community in Reddit. Our study began with the observation that most research on sustained waves of newcomers focuses on the destructive effect of newcomers and frequently invokes Usenet s infamous Eternal September. Our qualitative study argued that the /r/nosleep community managed its surge of newcomers gracefully through strategic preparation by moderators, technological systems to reign in on norm violations, and a shared sense of protecting the community s immersive environment among participants.
We are thrilled that, less a year after the publication of our study, Zhiyuan Jerry Lin and a group of researchers at Stanford have published a quantitative test of our study s findings! Lin analyzed 45 million comments and upvote patterns from 10 Reddit communities that a massive inundation of newcomers like the one we studied on /r/nosleep. Lin s group found that these communities retained their quality despite a slight dip in its initial growth period.
Our team discussed doing a quantitative study like Lin s at some length and our paper ends with a lament that our findings merely reflected, propositions for testing in future work. Lin s study provides exactly such a test! Lin et al. s results suggest that our qualitative findings generalize and that sustained influx of newcomers need not doom a community to a descent into an Eternal September. Through strong moderation and the use of a voting system, the subreddits analyzed by Lin appear to retain their identities despite the surge of new users.
There are always limits to research projects work quantitative and qualitative. We think the Lin s paper compliments ours beautifully, we are excited that Lin built on our work, and we re thrilled that our propositions seem to have held up!
This blog post was written with Charlie Kiene. Our paper about /r/nosleep, written with Charlie Kiene and Andr s Monroy-Hern ndez, was published in the Proceedings of CHI 2016 and is released as open access. Lin s paper was published in the Proceedings of ICWSM 2017 and is also available online.
Repeating what I did for the last Debian releases with the #newinwheezy and #newinjessie games it s time for the #newinstretch game:
Debian/stretch AKA Debian 9.0 will include a bunch of packages for people interested in digital forensics. The packages maintained within the Debian Forensics team which are new in the Debian/stretch release as compared to Debian/jessie (and ignoring jessie-backports):
bruteforce-salted-openssl: try to find the passphrase for files encrypted with OpenSSL
cewl: custom word list generator
dfdatetime/python-dfdatetime: Digital Forensics date and time library
dfvfs/python-dfvfs: Digital Forensics Virtual File System
dfwinreg: Digital Forensics Windows Registry library
libsmdev(-utils): storage media device access library
libsmraw(-utils): split RAW image format access library
libvhdi(-utils): Virtual Hard Disk image format access library
libvmdk(-utils): VMWare Virtual Disk format access library
libvshadow(-utils): Volume Shadow Snapshot format access library
libvslvm(-utils): Linux LVM volume system format access librar
plaso: super timeline all the things
pompem: Exploit and Vulnerability Finder
pytsk/python-tsk: Python Bindings for The Sleuth Kit
rekall(-core): memory analysis and incident response framework
unhide.rb: Forensic tool to find processes hidden by rootkits (was already present in wheezy but missing in jessie, available via jessie-backports though)
winregfs: Windows registry FUSE filesystem
Join the #newinstretch game and present packages and features which are new in Debian/stretch.
I finally bought my first raspberry pi to setup as a router and wifi access point.
It wasn't easy.
I first had to figure out what to buy. I think that was the hardest part.
I ended up with:
Raspberry PI 3 Model B A1.2GHz 64-bit quad-core ARMv8 CPU, 1GB RAM (Model number: RASPBERRYPI3-MODB-1GB)
Transcend USB 3.0 SDHC / SDXC / microSDHC / SDXC Card Reader, TS-RDF5K (Black). I only needed this because I don't have one already and I will need a way to copy a raspbian image from my laptop to a micro SD card.
Centon Electronics Micro SD Card 16 GB (S1-MSDHC4-16G). This is the micro sd card.
Smraza Clear case for Raspberry Pi 3 2 Model B with Power Supply,2pcs Heatsinks and Micro USB with On/Off Switch. And this is the box to put it all in.
I already have a cable matters USB to ethernet device, which will provide the second ethernet connection so this device can actually work as a router.
I studiously followed the directions to download the raspbian image and copy it to my micro sd card. I also touched a file on the boot partition called ssh so ssh would start automatically. Note: I first touched the ssh file on the root partition (sdb2) before realizing it belonged on the boot partition (sdb1). And, despite ambiguous directions found on the Internet, lowercase 'ssh' for the filename seems to do the trick.
Then, I found the IP address with the help of NMAP (sudo nmap -sn 192.168.69.*) and tried to ssh in but alas...
Connection reset by 192.168.69.116 port 22
No dice.
So, I re-mounted the sdb2 partition of the micro sd card and looked in var/log/auth.log and found:
May 5 19:23:00 raspberrypi sshd[760]: error: Could not load host key: /etc/ssh/ssh_host_ed25519_key
May 5 19:23:00 raspberrypi sshd[760]: fatal: No supported key exchange algorithms [preauth]
May 5 19:23:07 raspberrypi sshd[762]: error: key_load_public: invalid format
May 5 19:23:07 raspberrypi sshd[762]: error: Could not load host key: /etc/ssh/ssh_host_rsa_key
May 5 19:23:07 raspberrypi sshd[762]: error: key_load_public: invalid format
May 5 19:23:07 raspberrypi sshd[762]: error: Could not load host key: /etc/ssh/ssh_host_dsa_key
May 5 19:23:07 raspberrypi sshd[762]: error: key_load_public: invalid format
May 5 19:23:07 raspberrypi sshd[762]: error: Could not load host key: /etc/ssh/ssh_host_ecdsa_key
May 5 19:23:07 raspberrypi sshd[762]: error: key_load_public: invalid format
How did that happen? And wait a minute...
0 jamie@turkey:~$ ls -l /mnt/etc/ssh/ssh_host_ecdsa_key
-rw------- 1 root root 0 Apr 10 05:58 /mnt/etc/ssh/ssh_host_ecdsa_key
0 jamie@turkey:~$ date
Fri May 5 15:44:15 EDT 2017
0 jamie@turkey:~$
Are the keys embedded in the image? Isn't that wrong?
I fixed with:
NOTE: I just did a second installation and this didn't happen. Maybe something went wrong as I experiment with SSH vs ssh on the boot partition?
Then I could ssh in. I removed the pi user account and added my ssh key to /root/.ssh/authorized_keys and put a new name "mondragon" in the /etc/hostname file.
And... I upgraded to Debian stretch and rebooted.
Then, I followed these instructions for fixing the wifi (replacing the firmware does still work for me).
I plugged my cable matters USB/Ethernet adapter into the device so it would be recognized, but left it dis-connected.
Next I started to configure the device to be a wifi access point using this excellend tutorial, but decided I wanted to setup my networks using systemd-networkd instead.
Since /etc/network/interaces already had eth0 set to manual (because apparently it is controlled by dhcpcd instead), I didn't need any modifications there.
However, I wanted to use the dhcp client built-in to systemd-networkd, so to prevent dhcpcd from obtaining an IP address, I purged dhcpcd:
apt-get purge dhcpcd5
I was planning to also use systemd-networkd to name the devices (using *.link files) but nothing I could do could convince systemd to rename them, so I gave up and added /etc/udev/rules.d/70-persistent-net.rules:
Sadly, IPMasquerade doesn't seem to work either for some reason, so...
root@mondragon:~# cat /etc/systemd/system/masquerade.service
[Unit]
Description=Start masquerading because Masquerade=yes not working in wan.network.
[Service]
Type=oneshot
ExecStart=/sbin/iptables -t nat -A POSTROUTING -o wan -j MASQUERADE
[Install]
WantedBy=network.target
root@mondragon:~#
And, systemd DHCPServer worked, but then it didn't and I couldn't figure out how to debug, so...
apt-get install dnsmasq
Followed by:
root@mondragon:~# cat /etc/dnsmasq.d/mondragon.conf
# Don't provide DNS services (unbound does that).
port=0
interface=lan
interface=wlan
# Only provide dhcp services since systemd-networkd dhcpserver seems
# flakey.
dhcp-range=set:cable,192.168.69.100,192.168.69.150,255.255.255.0,4h
dhcp-option=tag:cable,option:dns-server,192.168.69.1
dhcp-option=tag:cable,option:router,192.168.69.1
dhcp-range=set:wifi,10.0.69.100,10.0.69.150,255.255.255.0,4h
dhcp-option=tag:wifi,option:dns-server,10.0.69.1
dhcp-option=tag:wifi,option:router,10.0.69.1
root@mondragon:~#
It would probably be simpler to have dnsmasq provide DNS service also, but I happen to like unbound:
apt-get install unbound
And...
root@mondragon:~# cat /etc/unbound/unbound.conf.d/server.conf
server:
interface: 127.0.0.1
interface: 192.168.69.1
interface: 10.0.69.1
access-control: 192.168.69.0/24 allow
access-control: 10.0.69.0/24 allow
# We do query localhost for our stub zone: loc.cx
do-not-query-localhost: no
# Up this level when debugging.
log-queries: no
logfile: ""
#verbosity: 1
# Settings to work better with systemcd
do-daemonize: no
pidfile: ""
root@mondragon:~#
Now on to the wifi access point.
apt-get install hostapd
And the configuration file:
root@mondragon:~# cat /etc/hostapd/hostapd.conf
# This is the name of the WiFi interface we configured above
interface=wlan
# Use the nl80211 driver with the brcmfmac driver
driver=nl80211
# This is the name of the network
ssid=peacock
# Use the 2.4GHz band
hw_mode=g
# Use channel 6
channel=6
# Enable 802.11n
ieee80211n=1
# Enable WMM
wmm_enabled=1
# Enable 40MHz channels with 20ns guard interval
ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40]
# Accept all MAC addresses
macaddr_acl=0
# Use WPA authentication
auth_algs=1
# Require clients to know the network name
ignore_broadcast_ssid=0
# Use WPA2
wpa=2
# Use a pre-shared key
wpa_key_mgmt=WPA-PSK
# The network passphrase
wpa_passphrase=xxxxxxxxxxxx
# Use AES, instead of TKIP
rsn_pairwise=CCMP
root@mondragon:~#
The hostapd package doesn't have a systemd start up file so I added one:
My last step was to modify /etc/ssh/sshd_config so it only listens on the lan and wlan interfaces (listening on wlan is a bit of a risk, but also useful when mucking with the lan network settings to ensure I don't get locked out).
Corrected a subtle bug in my django-staticfiles-dotd "staticfiles" library where the Content-Length HTTP header was calculated incorrectly in the presence of Unicode characters resulting in truncated output. (#2)
Various fixes to django-slack, a library to easily post messages to the Slack group-messaging utility from projects using the Django web development framework:
Don't require an explicit backend import when using the Celery task queue backend. (#41)
Actually generate and send messages asynchronously when using the Celery backend. (#44)
Fixed an issue with my local-debian-mirror tool where the option to disable DEP-11 mirroring wasn't working. (#1)
Fixed an issue in django-hipchat, a library to easily post messages to the Hipchat group-messaging utility from projects using the Django web development framework where the templates were not includes when installing via PyPI. (#1)
Created a quick-and-dirty tool to scrape a Squarespace blog and convert it to a PDF so I can read them on my Kindle e-reader. (tree)
Updated django-keyerror a library to post exceptions to the KeyError.com error tracking service to silence an AttributeError exception in some error-reporting edge-cases. (commit)
Suggested an improvement to the documentation for the upcoming Twitter Bootstrap version for the deprecated .hidden and .show CSS classes. (#19789)
Submitted a documentation update to the Ansible sever configuration tool's ufw firewall module. (commit)
What happened about the reproducible
builds effort this week:
Media coverage
Daniel Stender published an English translation of the article which originally
appeared in Linux Magazin in Admin Magazine.
Toolchain fixes
Fixes landed in the Debian archive:
Lunar uploaded docbook-to-man/1:2.0.0-34 which removes a timestamp in generated manpages. Original patch by Chris Lamb.
Stefano Rivera uploaded dh-python/1.20150628-1 which now sorts namespace files. Original patch by Chris Lamb.
Christian Hofstaedtler uploaded ruby2.2/2.2.2-2 which now uses UTC for the dates in gemspec files. Original patch by Chris Lamb.
reproducible.debian.net
A new package set for the X Strike Force has been added. (h01ger)
Bugs tagged with locale are now visible in the statistics. (h01ger)
Some work has been done add tests for NetBSD. (h01ger)
Many changes by Mattia Rizzolo have been merged on the whole infrastructure:
IRC notifications when known reproducible packages stops buildig successfully.
In a new paper, recently published in the open access journal PLOSONE, Aaron Shaw and I build on new research in survey methodology to describe a method for estimating bias in opt-in surveys of contributors to online communities. We use the technique to reevaluate the most widely cited estimate of the gender gap in Wikipedia.
A series of studies have shown that Wikipedia s editor-base is overwhelmingly male. This extreme gender imbalance threatens to undermine Wikipedia s capacity to produce high quality information from a full range of perspectives. For example, many articles on topics of particular interest to women tend to be under-produced or of poor quality.
Given the open and often anonymous nature of online communities, measuring contributor demographics is a challenge. Most demographic data on Wikipedia editors come from opt-in surveys where people respond to open, public invitations. Unfortunately, very few people answer these invitations. Results from opt-in surveys are unreliable because respondents are rarely representative of the community as a whole. The most widely-cited estimate from a large 2008 survey by the Wikimedia Foundation (WMF) and UN University in Maastrict (UNU-MERIT) suggested that only 13% of contributors were female. However, the very same survey suggested that less than 40% of Wikipedia s readers were female. We know, from several reliable sources, that Wikipedia s readership is evenly split by gender a sign of bias in the WMF/UNU-MERIT survey.
In our paper, we combine data from a nationally representative survey of the US by the Pew Internet and American Life Project with the opt-in data from the 2008 WMF/UNU-MERIT survey to come up with revised estimates of the Wikipedia gender gap. The details of the estimation technique are in the paper, but the core steps are:
We use the Pew dataset to provide baseline information about Wikipedia readers.
We apply a statistical technique called propensity scoring to estimate the likelihood that a US adult Wikipedia reader would have volunteered to participate in the WMF/UNU-MERIT survey.
We follow a process originally developed by Valliant and Dever to weight the WMF/UNU-MERIT survey to correct for estimated bias.
We extend this weighting technique to Wikipedia editors in the WMF/UNU data to produce adjusted estimates of the demographics of their sample.
Using this method, we estimate that the proportion of female US adult editors was 27.5% higher than the original study reported (22.7%, versus 17.8%), and that the total proportion of female editors was 26.8% higher (16.1%, versus 12.7%). These findings are consistent with other work showing that opt-in surveys tend to undercount women.
Overall, these results reinforce the basic substantive finding that women are vastly under-represented among Wikipedia editors.
Beyond Wikipedia, our paper describes a method online communities can adopt to estimate contributor demographics using opt-in surveys, but that is more credible than relying entirely on opt-in data. Advertising-intelligence firms like ComScore and Quantcast provide demographic data on the readership of an enormous proportion of websites. With these sources, almost any community can use our method (and source code) to replicate a similar analysis by: (1) surveying a community s readers (or a random subset) with the same instrument used to survey contributors; (2) combining results for readers with reliable demographic data about the readership population from a credible source; (3) reweighting survey results using the method we describe.
Although our new estimates will not help us us close the gender gap in Wikipedia or address its troubling implications, they give us a better picture of the problem. Additionally, our method offers an improved tool to build a clearer demographic picture of other online communities in general.
Fellow geeks of the
Canonical community team will be doing something pretty weird
this week. They're fund-raising for charity, for causes ranging
from environment to autism, from homeless support to kids
education, from poverty fight to water supplies. But that's not
weird. What's weird is that the fund-raising will culminate in a
24-hour work marathon
stream live, which will kick off tomorrow (Thursday) at
10:00 UTC.
As I like charity, and as I like contributing to Free Software,
I gladly accepted to rely here a challenge to the Debian
community by one of the marathon "horsemen", Michael
Hall:
if Michael receives at least 5 Debian-tagged
donations for the cause he has chosen to support (Autism Research Trust -
donate),
he'll do his marathon day wearing a Debian t-shirt
(which apparently is also the first Free Software t-shirt he has
ever owned).
I've already donated, so that makes only 4 donations to go to
add some Debian promotion salt to an Ubuntu marathon . To "tag"
your donation as originating from the Debian community, just leave
a message containing the word "debian" at the end of the donation
process.
The work Michael will be doing tomorrow is not entirely
Debian-specific, but Michael is Canonical's responsible for
relationship with upstream. I've worked with him in the past on a
range of topics, and I can testify that part of his work is
generally useful to Debian.
Raising the stakes, Nick Skaggs has decided to propose a similar
challenge:
for every 5 Debian-tagged donations to Nick's
cause (WaterAid - donate),
Nick will write a manpage for a Debian package that is in dire need
of one.
As above, just mention "debian" in your post-donation message to
tag your donation as originating from the Debian community.
Wanna take the challenge?
And how about the other 4 horsemen? No challenges to the Debian
community? Feel free to leave a comment and I'll raise the stakes
even more, updating the list above.
Update 4/10/2012, 13:35 +0200: Daniel Holbach
added his own challenge to the Debian community:
for every Debian-tagged donation to Daniel's
cause (Oxfam
- donate),
Daniel will (triage and) forward a patch from
Ubuntu to the Debian BTS.
I know I wrote about Android already today, but there is another thing that concerns me right now. I am owner of an Android-based phone (an HTC Dream) and recently switched my mobile network provider. The problem is that my new provider is a virtual provider and as such there is no real network of that provider. Now Android has a feature to turn off broadband connections when in roaming mode, which itself is a great idea and can save you from paying quite a lot of money when the phone connects to 3G abroad, but this feature also turns off broadband connections when roaming locally. All this is being discussed in bug report #3499.
After noticing this problem I became curious on how Android detects that it is roaming and I found the GsmServiceStateTracker.isRoamingBetweenOperators method to be responsible for that magic, but soon noticed that the method is not only inefficient, but also doesn t work as intended. This is hardly related to the bug mentioned above, but let s have a look at the code in question:
/**
* Set roaming state when gsmRoaming is true and, if operator mcc is the
* same as sim mcc, ons is different from spn
* @param gsmRoaming TS 27.007 7.2 CREG registered roaming
* @param s ServiceState hold current ons
* @return true for roaming state set
*/
private
boolean isRoamingBetweenOperators(boolean gsmRoaming, ServiceState s)
String spn = SystemProperties.get(PROPERTY_ICC_OPERATOR_ALPHA, "empty");
String onsl = s.getOperatorAlphaLong();
String onss = s.getOperatorAlphaShort();
boolean equalsOnsl = onsl != null && spn.equals(onsl);
boolean equalsOnss = onss != null && spn.equals(onss);
String simNumeric = SystemProperties.get(PROPERTY_ICC_OPERATOR_NUMERIC, "");
String operatorNumeric = s.getOperatorNumeric();
boolean equalsMcc = true;
try
equalsMcc = simNumeric.substring(0, 3).
equals(operatorNumeric.substring(0, 3));
catch (Exception e)
return gsmRoaming && !(equalsMcc && (equalsOnsl equalsOnss));
Okay, let me summarize what this piece of code does wrong, at least from my understanding:
It takes both the network operator alphanumeric identifier and alphanumeric long identifier and compares both to the alphanumeric identifier coming from the SIM card, whilst
it could simply use the network and SIM card numeric identifiers and compare those, which should be a lot cheaper than comparing those strings
Then it takes the first three characters/digits of the numeric identifiers (which indicate the country) and compares those
Now in my case my SIM card doesn t seem to provide the phone with a alphanumeric identifier, so the first two comparisons always fail for obvious reasons and, looking at the inline-if in the last line of that method my phone will always indicate that I am in roaming mode, even when I am not.
The problem is not only the logic which seems to be wrong, but I rather see the inefficient comparisons used there to be a major problem in embedded systems like mobile phones. This is the first piece of Android code I have had a look at, but if all other code is as ugly and inefficient as these few lines Android really needs some major fixes. Related to this I have reported bug #4590 and forked the git repository in question over at github, to fix this method, should be a matter of 5 minutes.
The Debian Python Modules Team is discussing which DVCS to switch to from SVN. Ondrej Certik asked how to generate a list of commiters to the team s repository, so I looked at it and got this:
emilio@saturno:~/deb/python-modules$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
865 piotr
609 morph
598 kov
532 bzed
388 pox
302 arnau
253 certik
216 shlomme
212 malex
175 hertzog
140 nslater
130 kobold
123 nijel
121 kitterma
106 bernat
99 kibi
87 varun
83 stratus
81 nobse
81 netzwurm
78 azatoth
76 mca
73 dottedmag
70 jluebbe
68 zack
68 cgalisteo
61 speijnik
61 odd_bloke
60 rganesan
55 kumanna
52 werner
50 haas
48 mejo
45 ucko
43 pabs
42 stew
42 luciano
41 mithrandi
40 wardi
36 gudjon
35 jandd
34 smcv
34 brettp
32 jenner
31 davidvilla
31 aurel32
30 rousseau
30 mtaylor
28 thomasbl
26 lool
25 gaspa
25 ffm
24 adn
22 jmalonzo
21 santiago
21 appaji
18 goedson
17 toadstool
17 sto
17 awen
16 mlizaur
16 akumar
15 nacho
14 smr
14 hanska
13 tviehmann
13 norsetto
13 mbaldessari
12 stone
12 sharky
11 rainct
11 fabrizio
10 lash
9 rodrigogc
9 pcc
9 miriam
9 madduck
9 ftlerror
8 pere
8 crschmidt
7 ncommander
7 myon
7 abuss
6 jwilk
6 bdrung
6 atehwa
5 kcoyner
5 catlee
5 andyp
4 vt
4 ross
4 osrevolution
4 lamby
4 baby
3 sez
3 joss
3 geole
2 rustybear
2 edmonds
2 astraw
2 ana
1 twerner
1 tincho
1 pochu
1 danderson
As it s likely that the Python Applications Packaging Team will switch too to the same DVCS at the same time, here are the numbers for its repo:
emilio@saturno:~/deb/python-apps$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
401 nijel
288 piotr
235 gothicx
159 pochu
76 nslater
69 kumanna
68 rainct
66 gilir
63 certik
52 vdanjean
52 bzed
46 dottedmag
41 stani
39 varun
37 kitterma
36 morph
35 odd_bloke
29 pcc
29 gudjon
28 appaji
25 thomasbl
24 arnau
20 sc
20 andyp
18 jalet
15 gerardo
14 eike
14 ana
13 dfiloni
11 tklauser
10 ryanakca
10 nxvl
10 akumar
8 sez
8 baby
6 catlee
4 osrevolution
4 cody-somerville
2 mithrandi
2 cjsmo
1 nenolod
1 ffm
Here I m the 4th most committer
And while I was on it, I thought I could do the same for the GNOME and GStreamer teams:
emilio@saturno:~/deb/pkg-gnome$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
5357 lool
2701 joss
1633 slomo
1164 kov
825 seb128
622 jordi
621 jdassen
574 manphiz
335 sjoerd
298 mlang
296 netsnipe
291 grm
255 ross
236 ari
203 pochu
198 ondrej
190 he
180 kilian
176 alanbach
170 ftlerror
148 nobse
112 marco
87 jak
84 samm
78 rfrancoise
75 oysteigi
73 jsogo
65 svena
65 otavio
55 duck
54 jcurbo
53 zorglub
53 rtp
49 wasabi
49 giskard
42 tagoh
42 kartikm
40 gpastore
34 brad
32 robtaylor
31 xaiki
30 stratus
30 daf
26 johannes
24 sander-m
21 kk
19 bubulle
16 arnau
15 dodji
12 mbanck
11 ruoso
11 fpeters
11 dedu
11 christine
10 cpm
7 ember
7 drew
7 debotux
6 tico
6 emil
6 bradsmith
5 robster
5 carlosliu
4 rotty
4 diegoe
3 biebl
2 thibaut
2 ejad
1 naoliv
1 huats
1 gilir
emilio@saturno:~/deb/pkg-gstreamer$ svn log egrep "^r[0-9]+ cut -f2 -d sed s/-guest// sort uniq -c sort -n -r
891 lool
840 slomo
99 pnormand
69 sjoerd
27 seb128
21 manphiz
8 he
7 aquette
4 elmarco
1 fabian
Conclusions:
- Why do I have the full python-modules and pkg-gstreamer trees, if I have just one commit to DPMT, and don t even have commit access to the GStreamer team?
- If you don t want to seem like you have done less commits than you have actually done, don t change your alioth name when you become a DD (hint: pox-guest and piotr in python-modules are the same person)
- If the switch to a new VCS was based on a vote where you have one vote per commit, the top 3 commiters in pkg-gnome could win the vote if they chosed the same! For python-apps it s the 4 top commiters, and the 7 ones for python-modules. pkg-gstreamer is a bit special
Hey,
Xfce 4.6 Beta 2 Hopper has just been released and,
guess what, packages are already available. Still not in unstable
(or even experimental), that will wait post-lenny. You can install
it from my repository, and,
as usual, there's no Debian support on them.
The known_issue page
is currently quite empty, and this beta looks really quite
polished. Packages currently available are i386 and amd64, powerpc
will be available when I have time to put the mac at build.
This beta features mostly bug fixing, and some features too:
power management stuff in xfce4-session. Based on a patch
included long ago in Ubuntu, it means that when you click the
logout button it'll ask you if you want to suspend/hibernate the
box. That works through hal and it may need some tuning on the
machine. Reports appreciated here or there.
soname bump in xfconf. That means package renaming from
libxconf-0-0 to libxfconf-0-2. No standard upgrade path
(Replaces:) is provided because I don't want to clutter
the packages before upload to Debian. This will continue for
following changes (libxfconf-0-3 for rc for example). For the final
release the library name will be changed to libxfconf-1 and the
soname will be reset to 0, which means the package name will be
libxfconf-1-0. There again, no upgrade path will be provided for
people using my repository. The only upgraded path supported will
be 4.4 →4.6. (but you won't be totally alone in the dark, I
use 4.6 beta on my laptop so I'll pass through hell before
you)
keyboard shortcut can be configured (both window manager ones
and applications) and there is now conflict handling
session management has been greatly improved, you can now
save the session immediately (a long-wanted feature) and
it manages stuff like SMRestartImmediately etc.
The official announcement can be found
there.
Have fnu!
A reminder on why we do Free Software
Some months ago I needed to find out if proper forward-only cursors were
implemented in the ODBC driver for MySQL. I didn't go very far with the
documentation, but I've been diving in its source code and within 30 minutes I
knew everything there was to know about the
subject.
Now, after years I haven't touched proprietary software at all, I've been asked
to port dballe to use the
Oracle ODBC driver.
I had a similar problem: find out how to use scrollable cursors with Oracle's
ODBC driver. The release notes (a file called ODBCRelnotesUS.htm that I
probably can't put online without their consent) say that they're implemented
since ages, but
SQLSetStmtAttr always
tells me that the driver does not support it. Maybe I have to enable it
somewhere? Maybe it's a bug?
I could have found out with one hour browsing this driver's source code, but
that wasn't available. Instead, I had to spend a whole day of trial and error,
google searching with all sort of keyword combinations, looking at strings
output on the .so libraries, and in the end I still don't
know.
That porting project is now stuck indefinitely, and my customers had to pay for
a full day of fruitless work.
So, here's the reminder: If you base on 3rd party proprietary software, you
can't provide a serious service.
Venkatesh Hariharan recently wrote an article on Indias traditions of knowledge and how Open Source meshes with them. Now having met him at foss.in and knowing of his work, I am 100% convinced he is on the side of the angels and their is no malice in what he wrote. However, his interpretation (really not his, but the conventional wisdom amongst the Indian elite.) of Indian history and culture is so monumentally wrong it bears correction. While I admit my hackles were raised by the cultural and theological mistakes made, there are implications for Open Source advocacy in India and the world at large too.
He starts by giving the example of Nalanda, supposedly "the first university in the world." but more accurately a Buddhist vihara (monastery.) Its' very name indicated sharing. I use the past tense because Nalanda doesn't exist anymore. In fact except in a few border areas influenced by Tibet or Burma, Buddhism itself is so thoroughly extinct in the land of its birth, the place where it flourished for over 1500 years, that the only reason people like Hariharan or myself only know that there ever was such a thing due to the work of 19th century Western Indologists. How could this be? One theory is that a resurgent Hinduism coopted or persecuted the Buddhists into oblivion. However this is unsatisfactory. It does not explain why Jainism (a religion as old as Buddhism and equally heretical for exactly the same reasons from the Hindu point of view) still survives to this day and it doesn't explain how it could have happened for even the most advanced genocidal regimes of the 20th century have been unable to "disappear" undesirables to the extent which Buddhism disappeared.
A more plausible explanation revolves around the fact that, as I mentioned before, places like Nalanda were Viharas. This doesn't mean that they can't have university-like properties too; after all the great academic institutions of Europe also had religious foundations but the monastic nature of a vihara is salient. A monastery is not a place you go to engage the world, its a place you go to escape it. To mix metaphors a bit, the viharas were cathedrals not bazaars. While there may have been a lively intellectual life within its walls, there is no evidence that Nalanda and its denizens interacted much with the surrounding culture. An economically unproductive institution of such vast size (15,000 monks at its peak) could only survive by the patronage of kings and other rich people. When that patronage dried up as for example when the medieval Muslim invasions swept away the old royalty, Nalanda and its sisters couldn't survive. Just as when a proprietary software company dies, its "intellectual property" often dies with it, when centralized Buddhism died, Buddhist ideas died with it. (Luckily Buddhism had spread to other countries by then and developed new, less monolithic forms so it was not lost to the world altogether.)
In contrast Hinduism (and for that matter Jainism) while having all due respect for monks were and are primarily lay movements. The pursuit of morality (Dharma), wealth (artha) and pleasure (kama) in this world are equally valid human goals alongside liberation from the world (moksha.) and this manifested as the much maligned institution of caste. Hariharan quotes the conventional view probably familiar to Westerners from Anthropology 101, a lock-step hierarchy of four classes (varna is the Sanskrt word) with fixed occupations, with knowledge and privilege on top and ignorance and oppression at the bottom. The thing is, this view bears no relationship whatsoever with social reality in India now or historically. There is another Sanskrit word, jati, which can be translated as caste. There are thousands of these jatis and while Indian thinkers have periodically attempted to try and shoehorn them into the varna framework, they have had little success and varna is for all practical purposes just a theory. And this is not just a fall from some putative golden age (Like the canard that once caste was flexible and merit based and only later became rigid.) but the state of affairs throughout recorded history.
A jati consists of people who share a common (or fictive) kinship, who have particular food and ritual choices, and marry endogamously. While many jatis are associated with a particular occupation, many others are not. This is what caste means to a typical Indian (it is also the social organization prevalent amongst Jains, Sikhs, Muslims and Christians even if they don't accept its theological basis.) Yes, hierarchy and concommitantly oppression, are in some cases a byproduct of casteist thinking but they are hardly its raison d'etre. A comparison can be made to the Western nuclear family system. Some people think it is superior to any other form of social organization. Most people wouldn't go that far but go along with it as "the way it is." Some of those of a liberal persuasion might want tweaks of certain aspects of it. A very small fringe think it is nothing more than a nefarious plot to spread patriarchy, subjugation and racism etc. They are mostly considered kooks by the other groups. (Unfortunately when India and caste are concerned, the kooks are ruling the roost.) The diversity of jatis like the diversity of open source creates a certain resistence to disruption that a monoculture cannot. Thus though Hinduism also suffered grievously from invasions, it was able to hold on where Indian Buddhism could not.
Brahmin (sic. That's an English mispronounciation.) is also misinterpreted as one who knows Brahma (masculine noun. The Creator God.) instead of Brahman (neuter noun. The supreme spirit pervading the Universe.) though even then it would be wrong because Brahmin, Brahma, and Brahman all come from a common root word brhati (feminine noun. Sacred speech.) as any Sanskrit dictionary will attest.
Hariharan cites the example of the sacred scriptures (Sic. He means the Vedas) and the notorious statement of the Manusmrti saying that an unqualified person who hears them should have molten lead poured into their ears as examples of the proprietarization of knowledge. Now the long-suffering untouchables may have been afraid of many things but having lead earplugs was not one of them. There is no historical evidence that any such action was ever attempted. You can criticize the Manusmrti for even saying such a thing but that is a far cry from using it as evidence of historical attitudes. You see, the Manusmrti is not even in the most orthodox conception Hindu Sharia and the Vedas are not "Sacred Scriptures" in the sense that the Bible or Koran are. Rather they are part of a wide range of significant literary works respected in Hinduism of which other important examples are the two Epics the Mahabharata (which includes the Bhagavadgita), and the Ramayana, and the vast compendia of geneologies, philosophy, lore and legend called the Puranas. Hariharan actually missed a good chance to make his point here because while the Vedas are indeed "proprietary" the other works I mentioned are not and have in fact permeated all over India and even beyond its borders. Many Indian languages owe their literary existence to translation of the epics or Puranas. For instance one of the earliest monuments of Hindi literature is the Ramacharitamanasa, Saint Tulsidas's translation of the Ramayana. When he wrote it in the 16th century, he did face some opposition from those who didn't think such an august work should be rendered into a "vulgar" tongue, he also received encouragement from equally orthodox people (including those who believed the Vedas should be restricted to Brahmins) Today it is by many accounts the most popular Hindu holy book in existence. So really the whole idea that the Hindu canon shows a fall from open grace to rigidly controlled private property is just bogus.
To those who have slogged their way though all this verbiage wondering what it has do with Linux and Open Source, let me assure you I am getting to the point :-) It's bad enough from my point of view as a devout Hindu that my religion would be misrepresented this way but from my point of view as a supporter of Free Software ideals, it is catastrophic.
Linux and Open Source in India has spread, as in most parts of the world, primararily as a hobby amongst computer enthusiasts and for a hobby it's doing ok. However if you think of it as more than that, as a tool to enhance society and empower peoples lives, that is not nearly good enough. But how can one serve a society if one apparently has no clue about that societies history and mores? The developing world is littered with failed projects that were carried out with the purest of motives but no regard to what the putative clients actually wanted. Do we want Linux and Open Source deployments to go the same way?
For a couple of years now I've been trying to get Indian and NRI programmers to get involved in localization in debian-installer and elsewhere but even with the small minority that are sympathetic to Open Source (a scandal in itself.) it is like pulling teeth. To their credit, Red Hat India pays people to work on such things but volunteerism is the heart of the Open Source movement as we all know so the apathy of Indian IT professionals is deeply troubling. Hariharan is trying to increase interest by appealing to the Indians' sense of cultural pride which is a good tactic--it has been successfuly applied by many non-western or minority cultures--but he is not going to be very successful if the vision of Indian culture he is presenting is as alien as Eskimo culture to the vast majority of the people.
Western Open Source advocates also run into difficulties due to being tone-deaf in their understanding of how third-world cultures actually operate. When India declined to participate in the One Laptop Per Child (OLPC) project, some commentators on places like Slashdot were perplexed. Wouldn't a poor country want an inexpensive yet efficient pc using free software instead of high-priced imports? The thing they missed is that while people in India may be poor, thanks to a more-or-less democratic polity and more-or-less open communications, their frame of reference is no longer their village. Their trends are increasingly set in Mumbai, New York, Paris, and London. The forward-thinking amongst them know that computers are the key to the future. They also know Bill Gates is the richest man in the world thanks to something called "Windows" knowing which can get you a green card and your own chance of becoming rich. So if Linux comes along, presented as a cheap alternative which is "just as good", the fake Gucci handbag of IT if you will, it is not going to impress. Thinkpads and Dells appeal to the upwardly-mobile persons vanity, a third-world hand-cranked laptop does not. (This incidently is what is behind the "Nalanda university" meme too. Monasteries are yesterday. Universities are the in thing so we have to have some in our past and not just any but by golly, the first university in the world.)
When they venture from pure technology into the realm of culture, Open Source advocates need to be a lot more sophisticated, not to mention accurate, in their arguments lest they fall on deaf ears. Platitudes and sentimentality about "spiritual" versus "material" culture aren't enough. There must be concrete demonstrations of the benefits of why the Open Source approach to knowledge is better. They must be more sensitive in their presentation to avoid needlessly antagonizing people who might otherwise be natural allies. I believe the Open Source ideal is good for India and the whole world, not just in software but for society at large and I wish Venkatesh Hariharan and all the others on the frontlines of the fight for openness the best of luck. But good intentions cannot excuse sloppy thinking.
DB-ALLe has been released!
After more than a year of (paid) work, DB-ALLe has been released!
DB-All.e is a fast, temporary, on-disk database where meteorological data can
be stored, searched, retrieved and updated. To make computation easier, data
is stored as physical data, that is, as measures of a variable in a specific
point of space and time, rather than as a sequence of report.
it is temporary, to be used for a limited time and then be deleted.
does not need backup, since it only contains replicated or derived data.
write access is enabled for its users.
it is fast for both read and for write access.
it is based on physical principles, that is, the data it contains are
defined in terms of omogeneous and consistent physical data. For example, it
is impossible for two incompatible values to exist in the same point in
space and time.
it can manage fixed station and moving stations such as airplanes or ships.
it can manage both observational and forecast data.
it can manage data along all three dimensions in space, such as data from
soundings and airplanes.
it can work based on physical parameters or on report types.
DB-ALLe has been written by me on behalf of ARPA-SIM a public agency
which realises that being funded with public money also means letting the
public access the result of what they do:
Copyright (C) 2005,2006 ARPA-SIM <urpsim@smr.arpa.emr.it>
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 2 of the License
Oh, and they make an outstanding weather forecast, the best you can find for my
region. It's so good that people even started to trust them! :)
A real sad state of affairs 
Repeating what I did for the last Debian releases with the 
Here is my monthly update covering a large part of what I have been doing in the free software world (
What happened about the 
. To "tag"
your donation as originating from the Debian community, just leave
a message containing the word "debian" at the end of the donation
process.
The work Michael will be doing tomorrow is not entirely
Debian-specific, but Michael is Canonical's responsible for
relationship with upstream. I've worked with him in the past on a
range of topics, and I can testify that part of his work is
generally useful to Debian.
I know I wrote about Android already today, but there is another thing that concerns me right now. I am owner of an Android-based phone (an HTC Dream) and recently switched my mobile network provider. The problem is that my new provider is a 
The Debian Python Modules Team is 
And while I was on it, I thought I could do the same for the GNOME and GStreamer teams:
(hint: pox-guest and piotr in python-modules are the same person)
Hey,
Xfce 4.6 Beta 2 Hopper has just been released and,
guess what, packages are already available. Still not in unstable
(or even experimental), that will wait post-lenny. You can install
it from 