Search Results: "bootc"

18 February 2021

Jonathan McDowell: Hacking and Bricking the EE Opsrey 2 Mini

I ve mentioned in the past my twisted EE network setup from when I moved in to my current house. The 4GEE WiFi Mini (also known as the EE Osprey 2 Mini or the EE40VB, and actually a rebadged Alcatel Y853VB) has been sitting unused since then, so I figured I d see about trying to get a shell on it. TL;DR: Of course it s running Linux, there s a couple of test points internally which bring out the serial console, but after finding those and logging in I discovered it s running ADB on port 5555 quite happily available without authentication both via wifi and the USB port. So if you have physical or local network access, instant root shell. Well done, folks. And then I bricked it before I could do anything more interesting. There s a lack of information about this device out there - most of the links I can find are around removing the SIM lock - so I thought I d document the pieces I found just in case anyone else is trying to figure it out. It s based around a Qualcomm MDM9607 SoC, paired with 64M RAM and 256M NAND flash. Wifi is via an RTL8192ES. Kernel is 3.18.20. Busybox is v1.23.1. It s running dnsmasq but I didn t grab the version. Of course there s no source or offer of source provided. Taking it apart is fairly easy. There s a single screw to remove, just beside the SIM slot. The coloured rim can then be carefully pried away from the back, revealing the battery. There are then 4 screws in the corners which need removed in order to be able to lift out the actual PCB and gain access to the serial console test points. EE40VB PCB serial console test points My mistake was going poking around trying to figure out where the updates are downloaded from - I know I m running a slightly older release than what s current, and the device can do an automatic download + update. Top tip; don t run Jrdrecovery. It ll error on finding /cache/update.zip and wipe the main partition anyway. That ll leave you in a boot loop where the device boots the recovery partition which tries to install /cache/update.zip which of course still doesn t exist. So. Where next? First, I need to get the device into a state where I can actually do something other than watch it boot into recovery, fail to flash and reboot. Best guess at present is to try and get it to enter the Qualcomm EDL (Emergency Download) mode. That might be possible with a custom USB cable that grounds D+ on boot. Alternatively I need to probe some of the other test points on the PCB and see if grounding any of those helps enter EDL mode. I then need a suitable firehose OEM-signed programmer image. And then I need to actually get hold of a proper EE40VB firmware image, either via one of the OTA update files or possibly via an Alcatel ADSU image (though no idea how to get hold of one, other than by posting to a random GSM device forum and hoping for the kindness of strangers). More updates if/when I make progress
Qualcomm bootloader log
Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.2-00053
S - IMAGE_VARIANT_STRING=LAATANAZA
S - OEM_IMAGE_VERSION_STRING=linux3
S - Boot Config, 0x000002e1
B -    105194 - SBL1, Start
D -     61885 - QSEE Image Loaded, Delta - (451964 Bytes)
D -     30286 - RPM Image Loaded, Delta - (151152 Bytes)
B -    459330 - Roger:boot_jrd_oem_main
B -    461526 - Welcome to key_check_poweron!!!
B -    466436 - REG0x00, rc=47
B -    469120 - REG0x01, rc=1f
B -    472018 - REG0x02, rc=1c
B -    474885 - REG0x03, rc=47
B -    477782 - REG0x04, rc=b2
B -    480558 - REG0x05, rc=
B -    483272 - REG0x06, rc=9e
B -    486139 - REG0x07, rc=
B -    488854 - REG0x08, rc=a4
B -    491721 - REG0x09, rc=80
B -    494130 - bq24295_probe: vflt/vsys/vprechg=0mV/0mV/0mV, tprechg/tfastchg=0Min/0Min, [0C, 0C]
B -    511546 - come to calculate vol and temperature!!
B -    511637 - ##############battery_core_convert_vntc: NTC_voltage=1785690
B -    517280 - battery_core_convert_vntc: <-44C, 1785690uV>, present=0
B -    529358 - bq24295_set_current_limit: setting=0mA, mode=-1, input/fastchg/prechg/termchg=-1mA/0mA/0mA/0mA
B -    534360 - bq24295_set_charge_current, rc=0,reg_val=0,i=0
B -    539636 - bq24295_enable_charge: setting=0, chg_enable=-1, otg_enable=0
B -    546072 - bq24295_enable_charging: enable_charging=0
B -    552172 - bq24295_set_current_limit: setting=0mA, mode=-1, input/fastchg/prechg/termchg=-1mA/0mA/0mA/0mA
B -    561566 - bq24295_set_charge_current, rc=0,reg_val=0,i=0
B -    567056 - bq24295_enable_charge: setting=0, chg_enable=0, otg_enable=0
B -    579286 - come to calculate vol and temperature!!
B -    579378 - ##############battery_core_convert_vntc: NTC_voltage=1785777
B -    585539 - battery_core_convert_vntc: <-44C, 1785777uV>, present=0
B -    597617 - charge_main: battery is plugout!!
B -    597678 - Welcome to pca955x_probe!!!
B -    601063 - pca955x_probe: PCA955X probed successfully!
D -     27511 - APPSBL Image Loaded, Delta - (179348 Bytes)
B -    633271 - QSEE Execution, Start
D -       213 - QSEE Execution, Delta
B -    638944 - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Start writting JRD RECOVERY BOOT
B -    650107 - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Start writting  RECOVERY BOOT
B -    653218 - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>read_buf[0] == 0
B -    659044 - SBL1, End
D -    556137 - SBL1, Delta
S - Throughput, 2000 KB/s  (782884 Bytes,  278155 us)
S - DDR Frequency, 240 MHz
littlekernel aboot log
Android Bootloader - UART_DM Initialized!!!
[0] welcome to lk
[0] SCM call: 0x2000601 failed with :fffffffc
[0] Failed to initialize SCM
[10] platform_init()
[10] target_init()
[10] smem ptable found: ver: 4 len: 17
[10] ERROR: No devinfo partition found
[10] Neither 'config' nor 'frp' partition found
[30] voltage of NTC  is 1789872!
[30] voltage of BAT  is 3179553!
[30] usb present is 1!
[30] Loading (boot) image (4171776): start
[530] Loading (boot) image (4171776): done
[540] DTB Total entry: 25, DTB version: 3
[540] Using DTB entry 0x00000129/00010000/0x00000008/0 for device 0x00000129/00010000/0x00010008/0
[560] JRD_CHG_OFF_FEATURE!
[560] come to jrd_target_pause_for_battery_charge!
[570] power_on_status.hard_reset = 0x0
[570] power_on_status.smpl = 0x0
[570] power_on_status.rtc = 0x0
[580] power_on_status.dc_chg = 0x0
[580] power_on_status.usb_chg = 0x0
[580] power_on_status.pon1 = 0x1
[590] power_on_status.cblpwr = 0x0
[590] power_on_status.kpdpwr = 0x0
[590] power_on_status.bugflag = 0x0
[590] cmdline: noinitrd  rw console=ttyHSL0,115200,n8 androidboot.hardware=qcom ehci-hcd.park=3 msm_rtb.filter=0x37 lpm_levels.sleep_disabled=1  earlycon=msm_hsl_uart,0x78b3000  androidboot.serialno=7e6ba58c androidboot.baseband=msm rootfstype=ubifs rootflags=b
[620] Updating device tree: start
[720] Updating device tree: done
[720] booting linux @ 0x80008000, ramdisk @ 0x80008000 (0), tags/device tree @ 0x81e00000
Linux kernel console boot log
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 3.18.20 (linux3@linux3) (gcc version 4.9.2 (GCC) ) #1 PREEMPT Thu Aug 10 11:57:07 CST 2017
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c53c7d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine model: Qualcomm Technologies, Inc. MDM 9607 MTP
[    0.000000] Early serial console at I/O port 0x0 (options '')
[    0.000000] bootconsole [uart0] enabled
[    0.000000] Reserved memory: reserved region for node 'modem_adsp_region@0': base 0x82a00000, size 56 MiB
[    0.000000] Reserved memory: reserved region for node 'external_image_region@0': base 0x87c00000, size 4 MiB
[    0.000000] Removed memory: created DMA memory pool at 0x82a00000, size 56 MiB
[    0.000000] Reserved memory: initialized node modem_adsp_region@0, compatible id removed-dma-pool
[    0.000000] Removed memory: created DMA memory pool at 0x87c00000, size 4 MiB
[    0.000000] Reserved memory: initialized node external_image_region@0, compatible id removed-dma-pool
[    0.000000] cma: Reserved 4 MiB at 0x87800000
[    0.000000] Memory policy: Data cache writeback
[    0.000000] CPU: All CPU(s) started in SVC mode.
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 17152
[    0.000000] Kernel command line: noinitrd  rw console=ttyHSL0,115200,n8 androidboot.hardware=qcom ehci-hcd.park=3 msm_rtb.filter=0x37 lpm_levels.sleep_disabled=1  earlycon=msm_hsl_uart,0x78b3000  androidboot.serialno=7e6ba58c androidboot.baseband=msm rootfstype=ubifs rootflags=bulk_read root=ubi0:rootfs ubi.mtd=16
[    0.000000] PID hash table entries: 512 (order: -1, 2048 bytes)
[    0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
[    0.000000] Memory: 54792K/69632K available (5830K kernel code, 399K rwdata, 2228K rodata, 276K init, 830K bss, 14840K reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xc8800000 - 0xff000000   ( 872 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xc8000000   ( 128 MB)
[    0.000000]     modules : 0xbf000000 - 0xc0000000   (  16 MB)
[    0.000000]       .text : 0xc0008000 - 0xc07e6c38   (8060 kB)
[    0.000000]       .init : 0xc07e7000 - 0xc082c000   ( 276 kB)
[    0.000000]       .data : 0xc082c000 - 0xc088fdc0   ( 400 kB)
[    0.000000]        .bss : 0xc088fe84 - 0xc095f798   ( 831 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] mpm_init_irq_domain(): Cannot find irq controller for qcom,gpio-parent
[    0.000000] MPM 1 irq mapping errored -517
[    0.000000] Architected mmio timer(s) running at 19.20MHz (virt).
[    0.000011] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 3579139424256ns
[    0.007975] Switching to timer-based delay loop, resolution 52ns
[    0.013969] Switched to clocksource arch_mem_counter
[    0.019687] Console: colour dummy device 80x30
[    0.023344] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000)
[    0.033666] pid_max: default: 32768 minimum: 301
[    0.038411] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.044902] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.052445] CPU: Testing write buffer coherency: ok
[    0.057057] Setting up static identity map for 0x8058aac8 - 0x8058ab20
[    0.064242]
[    0.064242] **********************************************************
[    0.071251] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.077817] **                                                      **
[    0.084302] ** trace_printk() being used. Allocating extra memory.  **
[    0.090781] **                                                      **
[    0.097320] ** This means that this is a DEBUG kernel and it is     **
[    0.103802] ** unsafe for produciton use.                           **
[    0.110339] **                                                      **
[    0.116850] ** If you see this message and you are not debugging    **
[    0.123333] ** the kernel, report this immediately to your vendor!  **
[    0.129870] **                                                      **
[    0.136380] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.142865] **********************************************************
[    0.150225] MSM Memory Dump base table set up
[    0.153739] MSM Memory Dump apps data table set up
[    0.168125] VFP support v0.3: implementor 41 architecture 2 part 30 variant 7 rev 5
[    0.176332] pinctrl core: initialized pinctrl subsystem
[    0.180930] regulator-dummy: no parameters
[    0.215338] NET: Registered protocol family 16
[    0.220475] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.284034] cpuidle: using governor ladder
[    0.314026] cpuidle: using governor menu
[    0.344024] cpuidle: using governor qcom
[    0.355452] msm_watchdog b017000.qcom,wdt: wdog absent resource not present
[    0.361656] msm_watchdog b017000.qcom,wdt: MSM Watchdog Initialized
[    0.371373] irq: no irq domain found for /soc/pinctrl@1000000 !
[    0.381268] spmi_pmic_arb 200f000.qcom,spmi: PMIC Arb Version-2 0x20010000
[    0.389733] platform 4080000.qcom,mss: assigned reserved memory node modem_adsp_region@0
[    0.397409] mem_acc_corner: 0 <--> 0 mV
[    0.401937] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[    0.408966] hw-breakpoint: maximum watchpoint size is 8 bytes.
[    0.416287] __of_mpm_init(): MPM driver mapping exists
[    0.420940] msm_rpm_glink_dt_parse: qcom,rpm-glink compatible not matches
[    0.427235] msm_rpm_dev_probe: APSS-RPM communication over SMD
[    0.432977] smd_open() before smd_init()
[    0.437544] msm_mpm_dev_probe(): Cannot get clk resource for XO: -517
[    0.445730] smd_channel_probe_now: allocation table not initialized
[    0.453100] mdm9607_s1: 1050 <--> 1350 mV at 1225 mV normal idle
[    0.458566] spm_regulator_probe: name=mdm9607_s1, range=LV, voltage=1225000 uV, mode=AUTO, step rate=4800 uV/us
[    0.468817] cpr_efuse_init: apc_corner: efuse_addr = 0x000a4000 (len=0x1000)
[    0.475353] cpr_read_fuse_revision: apc_corner: fuse revision = 2
[    0.481345] cpr_parse_speed_bin_fuse: apc_corner: [row: 37]: 0x79e8bd327e6ba58c, speed_bits = 4
[    0.490124] cpr_pvs_init: apc_corner: pvs voltage: [1050000 1100000 1275000] uV
[    0.497342] cpr_pvs_init: apc_corner: ceiling voltage: [1050000 1225000 1350000] uV
[    0.504979] cpr_pvs_init: apc_corner: floor voltage: [1050000 1050000 1150000] uV
[    0.513125] i2c-msm-v2 78b8000.i2c: probing driver i2c-msm-v2
[    0.518335] i2c-msm-v2 78b8000.i2c: error on clk_get(core_clk):-517
[    0.524478] i2c-msm-v2 78b8000.i2c: error probe() failed with err:-517
[    0.531111] i2c-msm-v2 78b7000.i2c: probing driver i2c-msm-v2
[    0.536788] i2c-msm-v2 78b7000.i2c: error on clk_get(core_clk):-517
[    0.542886] i2c-msm-v2 78b7000.i2c: error probe() failed with err:-517
[    0.549618] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    0.555202] i2c-msm-v2 78b9000.i2c: error on clk_get(core_clk):-517
[    0.561374] i2c-msm-v2 78b9000.i2c: error probe() failed with err:-517
[    0.570613] msm-thermal soc:qcom,msm-thermal: msm_thermal:Failed reading node=/soc/qcom,msm-thermal, key=qcom,core-limit-temp. err=-22. KTM continues
[    0.583049] msm-thermal soc:qcom,msm-thermal: probe_therm_reset:Failed reading node=/soc/qcom,msm-thermal, key=qcom,therm-reset-temp err=-22. KTM continues
[    0.596926] msm_thermal:msm_thermal_dev_probe Failed reading node=/soc/qcom,msm-thermal, key=qcom,online-hotplug-core. err:-517
[    0.609370] sps:sps is ready.
[    0.613137] msm_rpm_glink_dt_parse: qcom,rpm-glink compatible not matches
[    0.619020] msm_rpm_dev_probe: APSS-RPM communication over SMD
[    0.625773] mdm9607_s2: 750 <--> 1275 mV at 750 mV normal idle
[    0.631584] mdm9607_s3_level: 0 <--> 0 mV at 0 mV normal idle
[    0.637085] mdm9607_s3_level_ao: 0 <--> 0 mV at 0 mV normal idle
[    0.643092] mdm9607_s3_floor_level: 0 <--> 0 mV at 0 mV normal idle
[    0.649512] mdm9607_s3_level_so: 0 <--> 0 mV at 0 mV normal idle
[    0.655750] mdm9607_s4: 1800 <--> 1950 mV at 1800 mV normal idle
[    0.661791] mdm9607_l1: 1250 mV normal idle
[    0.666090] mdm9607_l2: 1800 mV normal idle
[    0.670276] mdm9607_l3: 1800 mV normal idle
[    0.674541] mdm9607_l4: 3075 mV normal idle
[    0.678743] mdm9607_l5: 1700 <--> 3050 mV at 1700 mV normal idle
[    0.684904] mdm9607_l6: 1700 <--> 3050 mV at 1700 mV normal idle
[    0.690892] mdm9607_l7: 1700 <--> 1900 mV at 1700 mV normal idle
[    0.697036] mdm9607_l8: 1800 mV normal idle
[    0.701238] mdm9607_l9: 1200 <--> 1250 mV at 1200 mV normal idle
[    0.707367] mdm9607_l10: 1050 mV normal idle
[    0.711662] mdm9607_l11: 1800 mV normal idle
[    0.716089] mdm9607_l12_level: 0 <--> 0 mV at 0 mV normal idle
[    0.721717] mdm9607_l12_level_ao: 0 <--> 0 mV at 0 mV normal idle
[    0.727946] mdm9607_l12_level_so: 0 <--> 0 mV at 0 mV normal idle
[    0.734099] mdm9607_l12_floor_lebel: 0 <--> 0 mV at 0 mV normal idle
[    0.740706] mdm9607_l13: 1800 <--> 2850 mV at 2850 mV normal idle
[    0.746883] mdm9607_l14: 2650 <--> 3000 mV at 2650 mV normal idle
[    0.752515] msm_mpm_dev_probe(): Cannot get clk resource for XO: -517
[    0.759036] cpr_efuse_init: apc_corner: efuse_addr = 0x000a4000 (len=0x1000)
[    0.765807] cpr_read_fuse_revision: apc_corner: fuse revision = 2
[    0.771809] cpr_parse_speed_bin_fuse: apc_corner: [row: 37]: 0x79e8bd327e6ba58c, speed_bits = 4
[    0.780586] cpr_pvs_init: apc_corner: pvs voltage: [1050000 1100000 1275000] uV
[    0.787808] cpr_pvs_init: apc_corner: ceiling voltage: [1050000 1225000 1350000] uV
[    0.795443] cpr_pvs_init: apc_corner: floor voltage: [1050000 1050000 1150000] uV
[    0.803094] cpr_init_cpr_parameters: apc_corner: up threshold = 2, down threshold = 3
[    0.810752] cpr_init_cpr_parameters: apc_corner: CPR is enabled by default.
[    0.817687] cpr_init_cpr_efuse: apc_corner: [row:65] = 0x15000277277383
[    0.824272] cpr_init_cpr_efuse: apc_corner: CPR disable fuse = 0
[    0.830225] cpr_init_cpr_efuse: apc_corner: Corner[1]: ro_sel = 0, target quot = 631
[    0.837976] cpr_init_cpr_efuse: apc_corner: Corner[2]: ro_sel = 0, target quot = 631
[    0.845703] cpr_init_cpr_efuse: apc_corner: Corner[3]: ro_sel = 0, target quot = 899
[    0.853592] cpr_config: apc_corner: Timer count: 0x17700 (for 5000 us)
[    0.860426] apc_corner: 0 <--> 0 mV
[    0.864044] i2c-msm-v2 78b8000.i2c: probing driver i2c-msm-v2
[    0.869261] i2c-msm-v2 78b8000.i2c: error on clk_get(core_clk):-517
[    0.875492] i2c-msm-v2 78b8000.i2c: error probe() failed with err:-517
[    0.882225] i2c-msm-v2 78b7000.i2c: probing driver i2c-msm-v2
[    0.887775] i2c-msm-v2 78b7000.i2c: error on clk_get(core_clk):-517
[    0.893941] i2c-msm-v2 78b7000.i2c: error probe() failed with err:-517
[    0.900719] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    0.906256] i2c-msm-v2 78b9000.i2c: error on clk_get(core_clk):-517
[    0.912430] i2c-msm-v2 78b9000.i2c: error probe() failed with err:-517
[    0.919472] msm-thermal soc:qcom,msm-thermal: msm_thermal:Failed reading node=/soc/qcom,msm-thermal, key=qcom,core-limit-temp. err=-22. KTM continues
[    0.932372] msm-thermal soc:qcom,msm-thermal: probe_therm_reset:Failed reading node=/soc/qcom,msm-thermal,
key=qcom,therm-reset-temp err=-22. KTM continues
[    0.946361] msm_thermal:get_kernel_cluster_info CPU0 topology not initialized.
[    0.953824] cpu cpu0: dev_pm_opp_get_opp_count: device OPP not found (-19)
[    0.960300] msm_thermal:get_cpu_freq_plan_len Error reading CPU0 freq table len. error:-19
[    0.968533] msm_thermal:vdd_restriction_reg_init Defer vdd rstr freq init.
[    0.975846] cpu cpu0: dev_pm_opp_get_opp_count: device OPP not found (-19)
[    0.982219] msm_thermal:get_cpu_freq_plan_len Error reading CPU0 freq table len. error:-19
[    0.991378] cpu cpu0: dev_pm_opp_get_opp_count: device OPP not found (-19)
[    0.997544] msm_thermal:get_cpu_freq_plan_len Error reading CPU0 freq table len. error:-19
[    1.013642] qcom,gcc-mdm9607 1800000.qcom,gcc: Registered GCC clocks
[    1.019451] clock-a7 b010008.qcom,clock-a7: Speed bin: 4 PVS Version: 0
[    1.025693] a7ssmux: set OPP pair(400000000 Hz: 1 uV) on cpu0
[    1.031314] a7ssmux: set OPP pair(1305600000 Hz: 7 uV) on cpu0
[    1.038805] i2c-msm-v2 78b8000.i2c: probing driver i2c-msm-v2
[    1.043587] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.052935] i2c-msm-v2 78b8000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.062006] irq: no irq domain found for /soc/wcd9xxx-irq !
[    1.069884] i2c-msm-v2 78b7000.i2c: probing driver i2c-msm-v2
[    1.074814] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.083716] i2c-msm-v2 78b7000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.093850] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    1.098889] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.107779] i2c-msm-v2 78b9000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.167871] KPI: Bootloader start count = 24097
[    1.171364] KPI: Bootloader end count = 48481
[    1.175855] KPI: Bootloader display count = 3884474147
[    1.180825] KPI: Bootloader load kernel count = 16420
[    1.185905] KPI: Kernel MPM timestamp = 105728
[    1.190286] KPI: Kernel MPM Clock frequency = 32768
[    1.195209] socinfo_print: v0.10, id=297, ver=1.0, raw_id=72, raw_ver=0, hw_plat=8, hw_plat_ver=65536
[    1.195209]  accessory_chip=0, hw_plat_subtype=0, pmic_model=65539, pmic_die_revision=131074 foundry_id=0 serial_number=2120983948
[    1.216731] sdcard_ext_vreg: no parameters
[    1.220555] rome_vreg: no parameters
[    1.224133] emac_lan_vreg: no parameters
[    1.228177] usbcore: registered new interface driver usbfs
[    1.233156] usbcore: registered new interface driver hub
[    1.238578] usbcore: registered new device driver usb
[    1.244507] cpufreq: driver msm up and running
[    1.248425] ION heap system created
[    1.251895] msm_bus_fabric_init_driver
[    1.262563] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0 Power-on reason: Triggered from PON1 (secondary PMIC) and 'cold' boot
[    1.273747] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0: Power-off reason: Triggered from UVLO (Under Voltage Lock Out)
[    1.285430] input: qpnp_pon as /devices/virtual/input/input0
[    1.291246] PMIC@SID0: PM8019 v2.2 options: 3, 2, 2, 2
[    1.296706] Advanced Linux Sound Architecture Driver Initialized.
[    1.302493] Add group failed
[    1.305291] cfg80211: Calling CRDA to update world regulatory domain
[    1.311216] cfg80211: World regulatory domain updated:
[    1.317109] Switched to clocksource arch_mem_counter
[    1.334091] cfg80211:  DFS Master region: unset
[    1.337418] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp), (dfs_cac_time)
[    1.354087] cfg80211:   (2402000 KHz - 2472000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.361055] cfg80211:   (2457000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.370545] NET: Registered protocol family 2
[    1.374082] cfg80211:   (2474000 KHz - 2494000 KHz @ 20000 KHz), (N/A, 2000 mBm), (N/A)
[    1.381851] cfg80211:   (5170000 KHz - 5250000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.389876] cfg80211:   (5250000 KHz - 5330000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.397857] cfg80211:   (5490000 KHz - 5710000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.405841] cfg80211:   (5735000 KHz - 5835000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.413795] cfg80211:   (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 0 mBm), (N/A)
[    1.422355] TCP established hash table entries: 1024 (order: 0, 4096 bytes)
[    1.428921] TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
[    1.435192] TCP: Hash tables configured (established 1024 bind 1024)
[    1.441528] TCP: reno registered
[    1.444738] UDP hash table entries: 256 (order: 0, 4096 bytes)
[    1.450521] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[    1.456950] NET: Registered protocol family 1
[    1.462779] futex hash table entries: 256 (order: -1, 3072 bytes)
[    1.474555] msgmni has been set to 115
[    1.478551] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[    1.485041] io scheduler noop registered
[    1.488818] io scheduler deadline registered
[    1.493200] io scheduler cfq registered (default)
[    1.502142] msm_rpm_log_probe: OK
[    1.506717] msm_serial_hs module loaded
[    1.509803] msm_serial_hsl_probe: detected port #0 (ttyHSL0)
[    1.515324] AXI: get_pdata(): Error: Client name not found
[    1.520626] AXI: msm_bus_cl_get_pdata(): client has to provide missing entry for successful registration
[    1.530171] msm_serial_hsl_probe: Bus scaling is disabled                      [    1.074814] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.083716] i2c-msm-v2 78b7000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.093850] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    1.098889] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.107779] i2c-msm-v2 78b9000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.167871] KPI: Bootloader start count = 24097
[    1.171364] KPI: Bootloader end count = 48481
[    1.175855] KPI: Bootloader display count = 3884474147
[    1.180825] KPI: Bootloader load kernel count = 16420
[    1.185905] KPI: Kernel MPM timestamp = 105728
[    1.190286] KPI: Kernel MPM Clock frequency = 32768
[    1.195209] socinfo_print: v0.10, id=297, ver=1.0, raw_id=72, raw_ver=0, hw_plat=8, hw_plat_ver=65536
[    1.195209]  accessory_chip=0, hw_plat_subtype=0, pmic_model=65539, pmic_die_revision=131074 foundry_id=0 serial_number=2120983948
[    1.216731] sdcard_ext_vreg: no parameters
[    1.220555] rome_vreg: no parameters
[    1.224133] emac_lan_vreg: no parameters
[    1.228177] usbcore: registered new interface driver usbfs
[    1.233156] usbcore: registered new interface driver hub
[    1.238578] usbcore: registered new device driver usb
[    1.244507] cpufreq: driver msm up and running
[    1.248425] ION heap system created
[    1.251895] msm_bus_fabric_init_driver
[    1.262563] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0 Power-on reason: Triggered from PON1 (secondary PMIC) and 'cold' boot
[    1.273747] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0: Power-off reason: Triggered from UVLO (Under Voltage Lock Out)
[    1.285430] input: qpnp_pon as /devices/virtual/input/input0
[    1.291246] PMIC@SID0: PM8019 v2.2 options: 3, 2, 2, 2
[    1.296706] Advanced Linux Sound Architecture Driver Initialized.
[    1.302493] Add group failed
[    1.305291] cfg80211: Calling CRDA to update world regulatory domain
[    1.311216] cfg80211: World regulatory domain updated:
[    1.317109] Switched to clocksource arch_mem_counter
[    1.334091] cfg80211:  DFS Master region: unset
[    1.337418] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp), (dfs_cac_time)
[    1.354087] cfg80211:   (2402000 KHz - 2472000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.361055] cfg80211:   (2457000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.370545] NET: Registered protocol family 2
[    1.374082] cfg80211:   (2474000 KHz - 2494000 KHz @ 20000 KHz), (N/A, 2000 mBm), (N/A)
[    1.381851] cfg80211:   (5170000 KHz - 5250000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.389876] cfg80211:   (5250000 KHz - 5330000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.397857] cfg80211:   (5490000 KHz - 5710000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.405841] cfg80211:   (5735000 KHz - 5835000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.413795] cfg80211:   (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 0 mBm), (N/A)
[    1.422355] TCP established hash table entries: 1024 (order: 0, 4096 bytes)
[    1.428921] TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
[    1.435192] TCP: Hash tables configured (established 1024 bind 1024)
[    1.441528] TCP: reno registered
[    1.444738] UDP hash table entries: 256 (order: 0, 4096 bytes)
[    1.450521] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[    1.456950] NET: Registered protocol family 1
[    1.462779] futex hash table entries: 256 (order: -1, 3072 bytes)
[    1.474555] msgmni has been set to 115
[    1.478551] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[    1.485041] io scheduler noop registered
[    1.488818] io scheduler deadline registered
[    1.493200] io scheduler cfq registered (default)
[    1.502142] msm_rpm_log_probe: OK
[    1.506717] msm_serial_hs module loaded
[    1.509803] msm_serial_hsl_probe: detected port #0 (ttyHSL0)
[    1.515324] AXI: get_pdata(): Error: Client name not found
[    1.520626] AXI: msm_bus_cl_get_pdata(): client has to provide missing entry for successful registration
[    1.530171] msm_serial_hsl_probe: Bus scaling is disabled
[    1.535696] 78b3000.serial: ttyHSL0 at MMIO 0x78b3000 (irq = 153, base_baud = 460800 [    1.544155] msm_hsl_console_setup: console setup on port #0
[    1.548727] console [ttyHSL0] enabled
[    1.548727] console [ttyHSL0] enabled
[    1.556014] bootconsole [uart0] disabled
[    1.556014] bootconsole [uart0] disabled
[    1.564212] msm_serial_hsl_init: driver initialized
[    1.578450] brd: module loaded
[    1.582920] loop: module loaded
[    1.589183] sps: BAM device 0x07984000 is not registered yet.
[    1.594234] sps:BAM 0x07984000 is registered.
[    1.598072] msm_nand_bam_init: msm_nand_bam_init: BAM device registered: bam_handle 0xc69f6400
[    1.607103] sps:BAM 0x07984000 (va:0xc89a0000) enabled: ver:0x18, number of pipes:7
[    1.616588] msm_nand_parse_smem_ptable: Parsing partition table info from SMEM
[    1.622805] msm_nand_parse_smem_ptable: SMEM partition table found: ver: 4 len: 17
[    1.630391] msm_nand_version_check: nand_major:1, nand_minor:5, qpic_major:1, qpic_minor:5
[    1.638642] msm_nand_scan: NAND Id: 0x1590aa98 Buswidth: 8Bits Density: 256 MByte
[    1.646069] msm_nand_scan: pagesize: 2048 Erasesize: 131072 oobsize: 128 (in Bytes)
[    1.653676] msm_nand_scan: BCH ECC: 8 Bit
[    1.657710] msm_nand_scan: CFG0: 0x290408c0,           CFG1: 0x0804715c
[    1.657710]             RAWCFG0: 0x2b8400c0,        RAWCFG1: 0x0005055d
[    1.657710]           ECCBUFCFG: 0x00000203,      ECCBCHCFG: 0x42040d10
[    1.657710]           RAWECCCFG: 0x42000d11, BAD BLOCK BYTE: 0x000001c5
[    1.684101] Creating 17 MTD partitions on "7980000.nand":
[    1.689447] 0x000000000000-0x000000140000 : "sbl"
[    1.694867] 0x000000140000-0x000000280000 : "mibib"
[    1.699560] 0x000000280000-0x000000e80000 : "efs2"
[    1.704408] 0x000000e80000-0x000000f40000 : "tz"
[    1.708934] 0x000000f40000-0x000000fa0000 : "rpm"
[    1.713625] 0x000000fa0000-0x000001000000 : "aboot"
[    1.718582] 0x000001000000-0x0000017e0000 : "boot"
[    1.723281] 0x0000017e0000-0x000002820000 : "scrub"
[    1.728174] 0x000002820000-0x000005020000 : "modem"
[    1.732968] 0x000005020000-0x000005420000 : "rfbackup"
[    1.738156] 0x000005420000-0x000005820000 : "oem"
[    1.742770] 0x000005820000-0x000005f00000 : "recovery"
[    1.747972] 0x000005f00000-0x000009100000 : "cache"
[    1.752787] 0x000009100000-0x000009a40000 : "recoveryfs"
[    1.758389] 0x000009a40000-0x00000aa40000 : "cdrom"
[    1.762967] 0x00000aa40000-0x00000ba40000 : "jrdresource"
[    1.768407] 0x00000ba40000-0x000010000000 : "system"
[    1.773239] msm_nand_probe: NANDc phys addr 0x7980000, BAM phys addr 0x7984000, BAM IRQ 164
[    1.781074] msm_nand_probe: Allocated DMA buffer at virt_addr 0xc7840000, phys_addr 0x87840000
[    1.791872] PPP generic driver version 2.4.2
[    1.801126] cnss_sdio 87a00000.qcom,cnss-sdio: CNSS SDIO Driver registered
[    1.807554] msm_otg 78d9000.usb: msm_otg probe
[    1.813333] msm_otg 78d9000.usb: OTG regs = c88f8000
[    1.820702] gbridge_init: gbridge_init successs.
[    1.826344] msm_otg 78d9000.usb: phy_reset: success
[    1.830294] qcom,qpnp-rtc qpnp-rtc-c7307000: rtc core: registered qpnp_rtc as rtc0
[    1.838474] i2c /dev entries driver
[    1.842459] unable to find DT imem DLOAD mode node
[    1.846588] unable to find DT imem EDLOAD mode node
[    1.851332] unable to find DT imem dload-type node
[    1.856921] bq24295-charger 4-006b: bq24295 probe enter
[    1.861161] qcom,iterm-ma = 128
[    1.864476] bq24295_otg_vreg: no parameters
[    1.868502] charger_core_register: Charger Core Version 5.0.0(Built at 20151202-21:36)!
[    1.877007] i2c-msm-v2 78b8000.i2c: msm_bus_scale_register_client(mstr-id:86):0x3 (ok)
[    1.885559] bq24295-charger 4-006b: bq24295_set_bhot_mode 3
[    1.890150] bq24295-charger 4-006b: power_good is 1,vbus_stat is 2
[    1.896588] bq24295-charger 4-006b: bq24295_set_thermal_threshold 100
[    1.902952] bq24295-charger 4-006b: bq24295_set_sys_min 3700
[    1.908639] bq24295-charger 4-006b: bq24295_set_max_target_voltage 4150
[    1.915223] bq24295-charger 4-006b: bq24295_set_recharge_threshold 300
[    1.922119] bq24295-charger 4-006b: bq24295_set_terminal_current_limit iterm_disabled=0, iterm_ma=128
[    1.930917] bq24295-charger 4-006b: bq24295_set_precharge_current_limit bdi->prech_cur=128
[    1.940038] bq24295-charger 4-006b: bq24295_set_safty_timer 0
[    1.945088] bq24295-charger 4-006b: bq24295_set_input_voltage_limit 4520
[    1.972949] sdhci: Secure Digital Host Controller Interface driver
[    1.978151] sdhci: Copyright(c) Pierre Ossman
[    1.982441] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.989092] sdhci_msm 7824900.sdhci: sdhci_msm_probe: ICE device is not enabled
[    1.995473] sdhci_msm 7824900.sdhci: No vreg data found for vdd
[    2.001530] sdhci_msm 7824900.sdhci: sdhci_msm_pm_qos_parse_irq: error -22 reading irq cpu
[    2.009809] sdhci_msm 7824900.sdhci: sdhci_msm_pm_qos_parse: PM QoS voting for IRQ will be disabled
[    2.018600] sdhci_msm 7824900.sdhci: sdhci_msm_pm_qos_parse: PM QoS voting for cpu group will be disabled
[    2.030541] sdhci_msm 7824900.sdhci: sdhci_msm_probe: sdiowakeup_irq = 353
[    2.036867] sdhci_msm 7824900.sdhci: No vmmc regulator found
[    2.042027] sdhci_msm 7824900.sdhci: No vqmmc regulator found
[    2.048266] mmc0: SDHCI controller on 7824900.sdhci [7824900.sdhci] using 32-bit ADMA in legacy mode
[    2.080401] Welcome to pca955x_probe!!
[    2.084362] leds-pca955x 3-0020: leds-pca955x: Using pca9555 16-bit LED driver at slave address 0x20
[    2.095400] sdhci_msm 7824900.sdhci: card claims to support voltages below defined range
[    2.103125] i2c-msm-v2 78b7000.i2c: msm_bus_scale_register_client(mstr-id:86):0x5 (ok)
[    2.114183] msm_otg 78d9000.usb: Avail curr from USB = 1500
[    2.120251] come to USB_SDP_CHARGER!
[    2.123215] Welcome to sn3199_probe!
[    2.126718] leds-sn3199 5-0064: leds-sn3199: Using sn3199 9-bit LED driver at slave address 0x64
[    2.136511] sn3199->led_en_gpio=21
[    2.139143] i2c-msm-v2 78b9000.i2c: msm_bus_scale_register_client(mstr-id:86):0x6 (ok)
[    2.150207] usbcore: registered new interface driver usbhid
[    2.154864] usbhid: USB HID core driver
[    2.159825] sps:BAM 0x078c4000 is registered.
[    2.163573] bimc-bwmon 408000.qcom,cpu-bwmon: BW HWmon governor registered.
[    2.171080] devfreq soc:qcom,cpubw: Couldn't update frequency transition information.
[    2.178513] coresight-fuse a601c.fuse: QPDI fuse not specified
[    2.184242] coresight-fuse a601c.fuse: Fuse initialized
[    2.192407] coresight-csr 6001000.csr: CSR initialized
[    2.197263] coresight-tmc 6026000.tmc: Byte Counter feature enabled
[    2.203204] sps:BAM 0x06084000 is registered.
[    2.207301] coresight-tmc 6026000.tmc: TMC initialized
[    2.212681] coresight-tmc 6025000.tmc: TMC initialized
[    2.220071] nidnt boot config: 0
[    2.224563] mmc0: new ultra high speed SDR50 SDIO card at address 0001
[    2.231120] coresight-tpiu 6020000.tpiu: NIDnT on SDCARD only mode
[    2.236440] coresight-tpiu 6020000.tpiu: TPIU initialized
[    2.242808] coresight-replicator 6024000.replicator: REPLICATOR initialized
[    2.249372] coresight-stm 6002000.stm: STM initialized
[    2.255034] coresight-hwevent 606c000.hwevent: Hardware Event driver initialized
[    2.262312] Netfilter messages via NETLINK v0.30.
[    2.266306] nf_conntrack version 0.5.0 (920 buckets, 3680 max)
[    2.272312] ctnetlink v0.93: registering with nfnetlink.
[    2.277565] ip_set: protocol 6
[    2.280568] ip_tables: (C) 2000-2006 Netfilter Core Team
[    2.285723] arp_tables: (C) 2002 David S. Miller
[    2.290146] TCP: cubic registered
[    2.293915] NET: Registered protocol family 10
[    2.298740] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    2.303407] sit: IPv6 over IPv4 tunneling driver
[    2.308481] NET: Registered protocol family 17
[    2.312340] bridge: automatic filtering via arp/ip/ip6tables has been deprecated. Update your scripts to load br_netfilter if you need this.
[    2.325094] Bridge firewalling registered
[    2.328930] Ebtables v2.0 registered
[    2.333260] NET: Registered protocol family 27
[    2.341362] battery_core_register: Battery Core Version 5.0.0(Built at 20151202-21:36)!
[    2.348466] pmu_battery_probe: vbat_channel=21, tbat_channel=17
[    2.420236] ubi0: attaching mtd16
[    2.723941] ubi0: scanning is finished
[    2.732997] ubi0: attached mtd16 (name "system", size 69 MiB)
[    2.737783] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    2.744601] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    2.751333] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    2.758540] ubi0: good PEBs: 556, bad PEBs: 2, corrupted PEBs: 0
[    2.764305] ubi0: user volume: 3, internal volumes: 1, max. volumes count: 128
[    2.771476] ubi0: max/mean erase counter: 192/64, WL threshold: 4096, image sequence number: 35657280
[    2.780708] ubi0: available PEBs: 0, total reserved PEBs: 556, PEBs reserved for bad PEB handling: 38
[    2.789921] ubi0: background thread "ubi_bgt0d" started, PID 96
[    2.796395] android_bind cdev: 0xC6583E80, name: ci13xxx_msm
[    2.801508] file system registered
[    2.804974] mbim_init: initialize 1 instances
[    2.809228] mbim_init: Initialized 1 ports
[    2.815074] rndis_qc_init: initialize rndis QC instance
[    2.819713] jrd device_desc.bcdDevice: [0x0242]
[    2.823779] android_bind scheduled usb start work: name: ci13xxx_msm
[    2.830230] android_usb gadget: android_usb ready
[    2.834845] msm_hsusb msm_hsusb: [ci13xxx_start] hw_ep_max = 32
[    2.840741] msm_hsusb msm_hsusb: CI13XXX_CONTROLLER_RESET_EVENT received
[    2.847433] msm_hsusb msm_hsusb: CI13XXX_CONTROLLER_UDC_STARTED_EVENT received
[    2.855851] input: gpio-keys as /devices/soc:gpio_keys/input/input1
[    2.861452] qcom,qpnp-rtc qpnp-rtc-c7307000: setting system clock to 1970-01-01 06:36:41 UTC (23801)
[    2.870315] open file error /usb_conf/usb_config.ini
[    2.876412] jrd_usb_start_work open file erro /usb_conf/usb_config.ini, retry_count:0
[    2.884324] parse_legacy_cluster_params(): Ignoring cluster params
[    2.889468] ------------[ cut here ]------------
[    2.894186] WARNING: CPU: 0 PID: 1 at /home/linux3/jrd/yanping.an/ee40/0810/MDM9607.LE.1.0-00130/apps_proc/oe-core/build/tmp-glibc/work-shared/mdm9607/kernel-source/drivers/cpuidle/lpm-levels-of.c:739 parse_cluster+0xb50/0xcb4()
[    2.914366] Modules linked in:
[    2.917339] CPU: 0 PID: 1 Comm: swapper Not tainted 3.18.20 #1
[    2.923171] [<c00132ac>] (unwind_backtrace) from [<c0011460>] (show_stack+0x10/0x14)
[    2.931092] [<c0011460>] (show_stack) from [<c001c6ac>] (warn_slowpath_common+0x68/0x88)
[    2.939175] [<c001c6ac>] (warn_slowpath_common) from [<c001c75c>] (warn_slowpath_null+0x18/0x20)
[    2.947895] [<c001c75c>] (warn_slowpath_null) from [<c034e180>] (parse_cluster+0xb50/0xcb4)
[    2.956189] [<c034e180>] (parse_cluster) from [<c034b6b4>] (lpm_probe+0xc/0x1d4)
[    2.963527] [<c034b6b4>] (lpm_probe) from [<c024857c>] (platform_drv_probe+0x30/0x7c)
[    2.971380] [<c024857c>] (platform_drv_probe) from [<c0246d54>] (driver_probe_device+0xb8/0x1e8)
[    2.980118] [<c0246d54>] (driver_probe_device) from [<c0246f30>] (__driver_attach+0x68/0x8c)
[    2.988467] [<c0246f30>] (__driver_attach) from [<c02455d0>] (bus_for_each_dev+0x6c/0x90)
[    2.996626] [<c02455d0>] (bus_for_each_dev) from [<c02465a4>] (bus_add_driver+0xe0/0x1c8)
[    3.004786] [<c02465a4>] (bus_add_driver) from [<c02477bc>] (driver_register+0x9c/0xe0)
[    3.012739] [<c02477bc>] (driver_register) from [<c080c3d8>] (lpm_levels_module_init+0x14/0x38)
[    3.021459] [<c080c3d8>] (lpm_levels_module_init) from [<c0008980>] (do_one_initcall+0xf8/0x1a0)
[    3.030217] [<c0008980>] (do_one_initcall) from [<c07e7d4c>] (kernel_init_freeable+0xf0/0x1b0)
[    3.038818] [<c07e7d4c>] (kernel_init_freeable) from [<c0582d48>] (kernel_init+0x8/0xe4)
[    3.046888] [<c0582d48>] (kernel_init) from [<c000dda0>] (ret_from_fork+0x14/0x34)
[    3.054432] ---[ end trace e9ec50b1ec4c8f73 ]---
[    3.059012] ------------[ cut here ]------------
[    3.063604] WARNING: CPU: 0 PID: 1 at /home/linux3/jrd/yanping.an/ee40/0810/MDM9607.LE.1.0-00130/apps_proc/oe-core/build/tmp-glibc/work-shared/mdm9607/kernel-source/drivers/cpuidle/lpm-levels-of.c:739 parse_cluster+0xb50/0xcb4()
[    3.083858] Modules linked in:
[    3.086870] CPU: 0 PID: 1 Comm: swapper Tainted: G        W      3.18.20 #1
[    3.093814] [<c00132ac>] (unwind_backtrace) from [<c0011460>] (show_stack+0x10/0x14)
[    3.101575] [<c0011460>] (show_stack) from [<c001c6ac>] (warn_slowpath_common+0x68/0x88)
[    3.109641] [<c001c6ac>] (warn_slowpath_common) from [<c001c75c>] (warn_slowpath_null+0x18/0x20)
[    3.118412] [<c001c75c>] (warn_slowpath_null) from [<c034e180>] (parse_cluster+0xb50/0xcb4)
[    3.126745] [<c034e180>] (parse_cluster) from [<c034b6b4>] (lpm_probe+0xc/0x1d4)
[    3.134126] [<c034b6b4>] (lpm_probe) from [<c024857c>] (platform_drv_probe+0x30/0x7c)
[    3.141906] [<c024857c>] (platform_drv_probe) from [<c0246d54>] (driver_probe_device+0xb8/0x1e8)
[    3.150702] [<c0246d54>] (driver_probe_device) from [<c0246f30>] (__driver_attach+0x68/0x8c)
[    3.159120] [<c0246f30>] (__driver_attach) from [<c02455d0>] (bus_for_each_dev+0x6c/0x90)
[    3.167285] [<c02455d0>] (bus_for_each_dev) from [<c02465a4>] (bus_add_driver+0xe0/0x1c8)
[    3.175444] [<c02465a4>] (bus_add_driver) from [<c02477bc>] (driver_register+0x9c/0xe0)
[    3.183398] [<c02477bc>] (driver_register) from [<c080c3d8>] (lpm_levels_module_init+0x14/0x38)
[    3.192107] [<c080c3d8>] (lpm_levels_module_init) from [<c0008980>] (do_one_initcall+0xf8/0x1a0)
[    3.200877] [<c0008980>] (do_one_initcall) from [<c07e7d4c>] (kernel_init_freeable+0xf0/0x1b0)
[    3.209475] [<c07e7d4c>] (kernel_init_freeable) from [<c0582d48>] (kernel_init+0x8/0xe4)
[    3.217542] [<c0582d48>] (kernel_init) from [<c000dda0>] (ret_from_fork+0x14/0x34)
[    3.225090] ---[ end trace e9ec50b1ec4c8f74 ]---
[    3.229667] /soc/qcom,lpm-levels/qcom,pm-cluster@0: No CPU phandle, assuming single cluster
[    3.239954] qcom,cc-debug-mdm9607 1800000.qcom,debug: Registered Debug Mux successfully
[    3.247619] emac_lan_vreg: disabling
[    3.250507] mem_acc_corner: disabling
[    3.254196] clock_late_init: Removing enables held for handed-off clocks
[    3.262690] ALSA device list:
[    3.264732]   No soundcard [    3.274083] UBIFS (ubi0:0): background thread "ubifs_bgt0_0" started, PID 102
[    3.305224] UBIFS (ubi0:0): recovery needed
[    3.466156] UBIFS (ubi0:0): recovery completed
[    3.469627] UBIFS (ubi0:0): UBIFS: mounted UBI device 0, volume 0, name "rootfs"
[    3.476987] UBIFS (ubi0:0): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[    3.486876] UBIFS (ubi0:0): FS size: 45838336 bytes (43 MiB, 361 LEBs), journal size 9023488 bytes (8 MiB, 72 LEBs)
[    3.497417] UBIFS (ubi0:0): reserved for root: 0 bytes (0 KiB)
[    3.503078] UBIFS (ubi0:0): media format: w4/r0 (latest is w4/r0), UUID 4DBB2F12-34EB-43B6-839B-3BA930765BAE, small LPT model
[    3.515582] VFS: Mounted root (ubifs filesystem) on device 0:12.
[    3.520940] Freeing unused kernel memory: 276K (c07e7000 - c082c000)
INIT: version 2.88 booting

16 April 2020

Rigved Rakshit: Intermediate Statistics

These are some of my notes on intermediate statistics from the Udemy Data Science Bootcamp. The Python code associated with this section is available here.

27 August 2017

Jonathan McDowell: On my way home from OMGWTFBBQ

I started writing this while sitting in Stansted on my way home from the annual UK Debian BBQ. I m finally home now, after a great weekend catching up with folk. It s a good social event for a bunch of Debian folk, and I m very grateful that Steve and Jo continue to make it happen. These days there are also a number of generous companies chipping in towards the cost of food and drink, so thanks also to Codethink and QvarnLabs AB for the food, Collabora and Mythic Beasts for the beer and Chris for the coffee. And Rob for chasing us all for contributions to cover the rest. I was trying to remember when the first one of these I attended was; trawling through mail logs there was a Cambridge meetup that ended up at Steve s old place in April 2001, and we ve consistently had the summer BBQ since 2004, but I m not clear on what happened in between. Nonetheless it s become a fixture in the calendar for those of us in the UK (and a number of people from further afield who regularly turn up). We ve become a bit more sedate, but it s good to always see a few new faces, drink some good beer (yay Milton), eat a lot and have some good conversations. This year also managed to get me a SheevaPlug so I could investigate #837989 - a bug with OpenOCD not being able to talk to the device. Turned out to be a channel configuration error in the move to new style FTDI support, so I ve got that fixed locally and pushed the one line fix upstream as well.

25 August 2017

Steve McIntyre: Let's BBQ again, like we did last summer!

It's that time again! Another year, another OMGWTFBBQ! We're expecting 50 or so Debian folks at our place in Cambridge this weekend, ready to natter, geek, socialise and generally have a good time. Let's hope the weather stays nice, but if not we have gazebo technology... :-) Many thanks to a number of awesome companies and people near and far who are sponsoring the important refreshments for the weekend: I've even been working on the garden this week to improve it ready for the event. If you'd like to come and haven't already told us, please add yourself to the wiki page!

31 May 2017

Lars Wirzenius: Using a Yubikey 4 for ensafening one's encryption

Introduction I've written before about using a U2F key with PAM. This post continues the theme and explains how to use a smartcard with GnuPG for storing OpenPGP private keys. Specifically, a Yubikey 4 card, because that's what I have, but any good GnuPG compatible card should work. The Yubikey is both a GnuPG compatible smart card, and a U2F card. The Yubikey 4 can handle keys up to 4096 bits. Older Yubikeys can only handle keys up to 2095 bits. The reason to do this is to make it harder for an attacker to steal your encryption keys. I will assume you don't already have an OpenPGP key, or are willing to generate a new one. I will also assume you run Debian stretch; some of the desktop environment setup details may differ between Debian versions or between Linux distributions. You will need: Terminology Some terminology: Outline The process outline is:
  1. Create a new, signing-only master key with GnuPG.
  2. Create three "subkeys", one each for encryption, signing, and authentication. These subkeys are what everyone else uses.
  3. Export copies of the master key pair and the subkey pairs and put them in a safe place.
  4. Put the subkeys on the Yubikey.
  5. GnuPG will automatically use the keys from the card. You have to have the card plugged into a USB port for things to work. If someone steals your laptop, they won't get the private subkeys. Even if they steal your Yubikey, they won't get them (the smartcard is physically designed to prevent that), and can't even use them (because there's PIN codes or passphrases and getting them wrong several times locks up the smartcard).
  6. Use gpg-agent as your SSH agent, and the authentication-only subkey on the Yubikey is used as your ssh key.
Configure GnuPG The process in more detail: Create new keys
$ gpg --full-generate-key
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
0 = key does not expire
= key expires in n days
w = key expires in n weeks
m = key expires in n months
y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:43:54 PM EEST
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: Lars Wirzenius
Email address: liw@liw.fi
Comment: test key
You selected this USER-ID:
"Lars Wirzenius (test key) <liw@liw.fi>>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: key 25FB738D6EE435F7 marked as ultimately trusted
gpg: directory '/home/liw/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/home/liw/.gnupg/openpgp-revocs.d/A734C10BF2DF39D19DC0F6C025FB738D6EE435F7.rev'
public and secret key created and signed.

Note that this key cannot be used for encryption. You may want to use
the command "--edit-key" to generate a subkey for this purpose.
pub rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid Lars Wirzenius (test key) <liw@liw.fi>
  • Note that I set a 1-year expiration for they key. The expiration can be extended at any time (if you have the master secret key), but unless you do, the key won't accidentally live longer than the chosen time.
  • Review the key:
$ gpg --list-secret-keys
/home/liw/.gnupg/pubring.kbx
----------------------------
sec rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid [ultimate] Lars Wirzenius (test key) <liw@liw.fi>
  • You now have the signing-only master key. You should now create three subkeys (keyid is the key identifier shown in the key listing, A734C10BF2DF39D19DC0F6C025FB738D6EE435F7 above). Use the --expert option to be able to add an authentication-only subkey.
$ gpg --edit-key --expert A734C10BF2DF39D19DC0F6C025FB738D6EE435F7z
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

sec rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
(3) DSA (sign only)
(4) RSA (sign only)
(5) Elgamal (encrypt only)
(6) RSA (encrypt only)
(7) DSA (set your own capabilities)
(8) RSA (set your own capabilities)
(10) ECC (sign only)
(11) ECC (set your own capabilities)
(12) ECC (encrypt only)
(13) Existing key
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
0 = key does not expire
= key expires in n days
w = key expires in n weeks
m = key expires in n months
y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:44:52 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
(3) DSA (sign only)
(4) RSA (sign only
(5) Elgamal (encrypt only)
(6) RSA (encrypt only)
(7) DSA (set your own capabilities)
(8) RSA (set your own capabilities)
(10) ECC (sign only)
(11) ECC (set your own capabilities)
(12) ECC (encrypt only)
(13) Existing key
Your selection? 6
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid
0 = key does not expire
= key expires in n days
w = key expires in n weeks
m = key expires in n months
y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:45:22 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
(3) DSA (sign only)
(4) RSA (sign only)
(5) Elgamal (encrypt only)
(6) RSA (encrypt only)
(7) DSA (set your own capabilities)
(8) RSA (set your own capabilities)
(10) ECC (sign only)
(11) ECC (set your own capabilities)
(12) ECC (encrypt only)
(13) Existing key
Your selection? 8

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Sign Encrypt

(S) Toggle the sign capability
(E) Toggle the encrypt capability
(A) Toggle the authenticate capability
(Q) Finished

Your selection? a

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Sign Encrypt Authenticate

(S) Toggle the sign capability
(E) Toggle the encrypt capability
(A) Toggle the authenticate capability
(Q) Finished

Your selection? s

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Encrypt Authenticate

(S) Toggle the sign capability
(E) Toggle the encrypt capability
(A) Toggle the authenticate capability
(Q) Finished

Your selection? e

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Authenticate

(S) Toggle the sign capability
(E) Toggle the encrypt capability
(A) Toggle the authenticate capability
(Q) Finished

Your selection? q
RSA keys may be btween 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
0 = key does not expire
= key expires in n days
w = key expires in n weeks
m = key expires in n months
y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:45:56 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> save
Export secret keys to files, make a backup
  • You now have a master key and three subkeys. They are hidden in the ~/.gnupg directory. It is time to "export" the secret keys out from there.
$ gpg --export-secret-key --armor keyid > master.key
$ gpg --export-secret-subkeys --armor keyid > subkeys.key
  • You should keep these files safe. You don't want to lose them, and you don't want anyone else to get access to them. I recommend you format two USB memory sticks, format them using full-disk encryption, and copy the exported files to both of them. Then keep them somewhere safe. There's ways of making this part more sophisticated, but that's for another time.
  • The next step involves some hoop-jumping. What we want is to have the master secret key NOT on you machine, so we tell GnuPG to remove it. We exported it above, so we won't lose it. However, deleting the master secret key also removes the secret subkeys. But we can import those without importing the master secret key.
$ gpg --delete-secret-key keyid
$ gpg --import subkeys.key
  • Now verify that you have the secret subkeys, but not the master key. There should be one line starting with sec# (note the hash mark, which indicates the key isn't available), and three lines starting with ssb (no hash mark).
$ gpg -K
/home/liw/.gnupg/pubring.kbx
----------------------------
sec# rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid [ultimate] Lars Wirzenius (test key) <liw@liw.fi>
ssb rsa4096 2017-05-29 [S] [expires: 2018-05-29]
ssb rsa4096 2017-05-29 [E] [expires: 2018-05-29]
ssb rsa4096 2017-05-29 [A] [expires: 2018-05-29]
Install subkeys on a Yubikey
  • Now insert the Yubikey in a USB slot. We can start transferring the secret subkeys to the Yubikey. If you want, you can set your name and other information, and change PIN codes. There's several types of PIN codes: normal use, unblocking a locked card, and a third PIN code for admin operations. Changing the PIN codes is a good idea, otherwise everyone will just try the default of 123456 (admin 12345678). However, I'm skipping that in the interest of brevity.
$ gpg -card-edit
...
  • Actually move the subkeys to the card. Note that this does a move, not a copy, and the subkeys will be removed from your ~/.gnupg (check with gpg -K).
$ gpg --edit-key liw
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb* rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb* rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb* rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(2) Encryption key
Your selection? 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb* rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 3

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb* rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(3) Authentication key
Your selection? 3

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb* rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> save
  • If you want to use several Yubikeys, or have a spare one just in case, repeat the previous four steps (starting from importing subkeys back into ~/.gnupg).
  • You're now done, as far GnuPG use is concerned. Any time you need to sign, encrypt, or decrypt something, GnuPG will look for your subkeys on the Yubikey, and will tell you to insert it in a USB port if it can't find the key.
Use subkey on Yubikey as your SSH key
  • To actually use the authentication-only subkey on the Yubikey for ssh, you need to configure your system to use gpg-agent as the SSH agent. Add the following line to .gnupg/gpg-agent.conf:
     enable-ssh-support
    
  • On a Debian stretch system with GNOME, edit /etc/xdg/autostart/gnome-keyring-ssh.desktop to have the following line, to prevent the GNOME ssh agent from starting up:
     Hidden=true
    
  • Edit /etc/X11/Xsession.options and remove or comment out the line that says use-ssh-agent. This stops a system-started ssh-agent from being started when the desktop start.
  • Create the file ~/.config/autostart/gpg-agent.desktop with the following content:
     [Desktop Entry]
     Type=Application
     Name=gpg-agent
     Comment=gpg-agent
     Exec=/usr/bin/gpg-agent --daemon
     OnlyShowIn=GNOME;Unity;MATE;
     X-GNOME-Autostart-Phase=PreDisplayServer
     X-GNOME-AutoRestart=false
     X-GNOME-Autostart-Notify=true
     X-GNOME-Bugzilla-Bugzilla=GNOME
     X-GNOME-Bugzilla-Product=gnome-keyring
     X-GNOME-Bugzilla-Component=general
     X-GNOME-Bugzilla-Version=3.20.0
    
  • To test, log out, and back in again, run the following in a terminal:
$ ssh-add -l
The output should contain a line that looks like this:
    4096 SHA256:PDCzyQPpd9tiWsELM8LwaLBsMDMm42J8/eEfezNgnVc cardno:000604626953 (RSA)
  • You need to export the authentication-only subkey in the SSH key format. You need this for adding to .ssh/authorized_keys, if nothing else.
$ gpg --export-ssh-key keyid > ssh.pub
  • Happy hacking.
See also See also the following links. I've used them to learn enough to write the above. Edited to fix:
  • Output of gpg -K after removing secret master key.

3 July 2016

Jonathan McDowell: Confirming all use of an SSH agent

For a long time I ve wanted an ssh-agent setup that would ask me before every use, so I could slightly more comfortably forward authentication over SSH without worrying that my session might get hijacked somewhere at the remote end (I often find myself wanting to pull authenticated git repos on remote hosts). I m at DebConf this week, which is an ideal time to dig further into these things, so I did so today. As is often the case it turns out this is already possible, if you know how. I began with a setup that was using GNOME Keyring to manage my SSH keys. This isn t quite what I want (eventually I want to get to the point that I can sometimes forward a GPG agent to remote hosts for signing purposes as well), so I set about setting up gpg-agent. I used Chris excellent guide to GnuPG/SSH Agent setup as a starting point and ended up doing the following:
$ echo use-agent >> ~/.gnupg/options
$ echo enable-ssh-support >> ~/.gnupg/gpg-agent.conf
$ sudo sed -i.bak "s/^use-ssh-agent/# use-ssh-agent/" /etc/X11/Xsession.options
$ sudo rm /etc/xdg/autostart/gnome-keyring-ssh.desktop
The first 2 commands setup my local agent, and told it to do SSH agent foo. The next stopped X from firing up ssh-agent, and the final one prevents GNOME Keyring from being configured to be the SSH agent, without having to remove libpam-gnome-keyring as Chris did. After the above I logged out of and into X again, and could see ~/.gnupg/S.gpg-agent.ssh getting created and env grep SSH showing SSH_AUTH_SOCK pointing to it (if GNOME Keyring is still handling things it ends up pointing to something like /run/user/1000/keyring/ssh). [Update: Luca Capello emailed to point out this was a bad approach; there s thankfully no need to do the last 2 commands that require root. #767341 removed the need to edit Xsession.options and you can prevent GNOME Keyring starting on a per user basis with:
(cat /etc/xdg/autostart/gnome-keyring-ssh.desktop ;
 echo 'X-GNOME-Autostart-enabled=false') > \
 ~/.config/autostart/gnome-keyring-ssh.desktop
] After this it turned out all I need to do was ssh-add -c <ssh keyfile>. The -c says confirm use and results in the confirm flag being appended to the end of ~/.gnupg/sshcontrol (so if you ve already done the ssh-add you can go and add the confirm if that s the behaviour you d like). Simple when you know how, but I ve had conversations with several people in the past who wanted the same thing and hadn t figured out how, so hopefully this is helpful to others.

30 June 2016

Chris Lamb: Free software activities in June 2016

Here is my monthly update covering a large part of what I have been doing in the free software world (previously):
Debian My work in the Reproducible Builds project was covered in our weekly reports. (#58, #59 & #60)
Debian LTS

This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Extended the lts-cve-triage.py script to ignore packages that are not subject to Long Term Support.

  • Issued DLA 512-1 for mantis fixing an XSS vulnerability.
  • Issued DLA 513-1 for nspr correcting a buffer overflow in a sprintf utility.
  • Issued DLA 515-1 for libav patching a memory corruption issue.
  • Issued DLA 524-1 for squidguard fixing a reflected cross-site scripting vulnerability.
  • Issued DLA 525-1 for gimp correcting a use-after-free vulnerability in the channel and layer properties parsing process.

Uploads
  • redis (2:3.2.1-1) New upstream bugfix release, plus subsequent upload to the backports repository.
  • python-django (1.10~beta1-1) New upstream experimental release.
  • libfiu (0.94-5) Misc packaging updates.


RC bugs

I also filed 170 FTBFS bugs against a7xpg, acepack, android-platform-dalvik, android-platform-frameworks-base, android-platform-system-extras, android-platform-tools-base, apache-directory-api, aplpy, appstream-generator, arc-gui-clients, assertj-core, astroml, bamf, breathe, buildbot, cached-property, calf, celery-haystack, charmtimetracker, clapack, cmake, commons-javaflow, dataquay, dbi, django-celery, django-celery-transactions, django-classy-tags, django-compat, django-countries, django-floppyforms, django-hijack, django-localflavor, django-markupfield, django-model-utils, django-nose, django-pipeline, django-polymorphic, django-recurrence, django-sekizai, django-sitetree, django-stronghold, django-taggit, dune-functions, elementtidy, epic4-help, fcopulae, fextremes, fnonlinear, foreign, fort77, fregression, gap-alnuth, gcin, gdb-avr, ggcov, git-repair, glance, gnome-twitch, gnustep-gui, golang-github-audriusbutkevicius-go-nat-pmp, golang-github-gosimple-slug, gprbuild, grafana, grantlee5, graphite-api, guacamole-server, ido, jless, jodreports, jreen, kdeedu-data, kdewebdev, kwalify, libarray-refelem-perl, libdbusmenu, libdebian-package-html-perl, libdevice-modem-perl, libindicator, liblrdf, libmail-milter-perl, libopenraw, libvisca, linuxdcpp, lme4, marble, mgcv, mini-buildd, mu-cade, mvtnorm, nose, octave-epstk, onioncircuits, opencolorio, parsec47, phantomjs, php-guzzlehttp-ringphp, pjproject, pokerth, prayer, pyevolve, pyinfra, python-asdf, python-ceilometermiddleware, python-django-bootstrap-form, python-django-compressor, python-django-contact-form, python-django-debug-toolbar, python-django-extensions, python-django-feincms, python-django-formtools, python-django-jsonfield, python-django-mptt, python-django-openstack-auth, python-django-pyscss, python-django-registration, python-django-tagging, python-django-treebeard, python-geopandas, python-hdf5storage, python-hypothesis, python-jingo, python-libarchive-c, python-mhash, python-oauth2client, python-proliantutils, python-pytc, python-restless, python-tidylib, python-websockets, pyvows, qct, qgo, qmidinet, quodlibet, r-cran-gss, r-cran-runit, r-cran-sn, r-cran-stabledist, r-cran-xml, rgl, rglpk, rkt, rodbc, ruby-devise-two-factor, ruby-json-schema, ruby-puppet-syntax, ruby-rspec-puppet, ruby-state-machine, ruby-xmlparser, ryu, sbd, scanlogd, signond, slpvm, sogo, sphinx-argparse, squirrel3, sugar-jukebox-activity, sugar-log-activity, systemd, tiles, tkrplot, twill, ucommon, urca, v4l-utils, view3dscene, xqilla, youtube-dl & zope.interface.

FTP Team

As a Debian FTP assistant I ACCEPTed 186 packages: akonadi4, alljoyn-core-1509, alljoyn-core-1604, alljoyn-gateway-1504, alljoyn-services-1504, alljoyn-services-1509, alljoyn-thin-client-1504, alljoyn-thin-client-1509, alljoyn-thin-client-1604, apertium-arg, apertium-arg-cat, apertium-eo-fr, apertium-es-it, apertium-eu-en, apertium-hbs, apertium-hin, apertium-isl, apertium-kaz, apertium-spa, apertium-spa-arg, apertium-tat, apertium-urd, arc-theme, argus-clients, ariba, beast-mcmc, binwalk, bottleneck, colorfultabs, dh-runit, django-modeltranslation, dq, dublin-traceroute, duktape, edk2, emacs-pdf-tools, eris, erlang-p1-oauth2, erlang-p1-sqlite3, erlang-p1-xmlrpc, faba-icon-theme, firefox-branding-iceweasel, golang-1.6, golang-defaults, golang-github-aelsabbahy-gonetstat, golang-github-howeyc-gopass, golang-github-oleiade-reflections, golang-websocket, google-android-m2repository-installer, googler, goto-chg-el, gr-radar, growl-for-linux, guvcview, haskell-open-browser, ipe, labplot, libalt-alien-ffi-system-perl, libanyevent-fcgi-perl, libcds-savot-java, libclass-ehierarchy-perl, libconfig-properties-perl, libffi-checklib-perl, libffi-platypus-perl, libhtml-element-library-perl, liblwp-authen-oauth2-perl, libmediawiki-dumpfile-perl, libmessage-passing-zeromq-perl, libmoosex-types-portnumber-perl, libmpack, libnet-ip-xs-perl, libperl-osnames-perl, libpodofo, libprogress-any-perl, libqtpas, librdkafka, libreoffice, libretro-beetle-pce-fast, libretro-beetle-psx, libretro-beetle-vb, libretro-beetle-wswan, libretro-bsnes-mercury, libretro-mupen64plus, libservicelog, libtemplate-plugin-datetime-perl, libtext-metaphone-perl, libtins, libzmq-ffi-perl, licensecheck, link-grammar, linux, linux-signed, lua-busted, magics++, mkalias, moka-icon-theme, neutron-vpnaas, newlisp, node-absolute-path, node-ejs, node-errs, node-has-flag, node-lodash-compat, node-strip-ansi, numba, numix-icon-theme, nvidia-graphics-drivers, nvidia-graphics-drivers-legacy-304xx, nvidia-graphics-drivers-legacy-340xx, obs-studio, opencv, pacapt, pgbackrest, postgis, powermock, primer3, profile-sync-daemon, pyeapi, pypandoc, pyssim, python-cutadapt, python-cymruwhois, python-fisx, python-formencode, python-hkdf, python-model-mommy, python-nanomsg, python-offtrac, python-social-auth, python-twiggy, python-vagrant, python-watcherclient, python-xkcd, pywps, r-bioc-deseq2, r-bioc-dnacopy, r-bioc-ensembldb, r-bioc-geneplotter, r-cran-adegenet, r-cran-adephylo, r-cran-distory, r-cran-fields, r-cran-future, r-cran-globals, r-cran-htmlwidgets, r-cran-listenv, r-cran-mlbench, r-cran-mlmrev, r-cran-pheatmap, r-cran-pscbs, r-cran-r.cache, refind, relatorio, reprotest, ring, ros-ros-comm, ruby-acts-as-tree, ruby-chronic-duration, ruby-flot-rails, ruby-numerizer, ruby-u2f, selenium-firefoxdriver, simgrid, skiboot, smtpping, snap-confine, snapd, sniffles, sollya, spin, subuser, superlu, swauth, swift-plugin-s3, syncthing, systemd-bootchart, tdiary-theme, texttable, tidy-html5, toxiproxy, twinkle, vmtk, wait-for-it, watcher, wcslib & xapian-core.

12 January 2016

Bits from Debian: New Debian Developers and Maintainers (November and December 2015)

The following contributors got their Debian Developer accounts in the last two months:
  • Stein Magnus Jodal (jodal)
  • Prach Pongpanich (prach)
  • Markus Koschany (apo)
  • Bernhard Schmidt (berni)
  • Uwe Kleine-K nig (ukleinek)
  • Timo Weing rtner (tiwe)
  • Sebastian Andrzej Siewior (bigeasy)
  • Mattia Rizzolo (mattia)
  • Alexandre Viau (aviau)
  • Lev Lamberov (dogsleg)
  • Adam Borowski (kilobyte)
  • Chris Boot (bootc)
The following contributors were added as Debian Maintainers in the last two months:
  • Alf Gaida
  • Andrew Ayer
  • Marcio de Souza Oliveira
  • Alexandre Detiste
  • Dave Hibberd
  • Andreas Boll
  • Punit Agrawal
  • Edward Betts
  • Shih-Yuan Lee
  • Ivan Udovichenko
  • Andrew Kelley
  • Benda Xu
  • Russell Sim
  • Paulo Roberto Alves de Oliveira
  • Marc Fournier
  • Scott Talbert
  • Sergio Durigan Junior
  • Guillaume Turri
  • Michael Lustfield
Congratulations!

31 March 2015

Dirk Eddelbuettel: R / Finance 2015 Open for Registration

The annoucement below just went to the R-SIG-Finance list. More information is as usual at the R / Finance page.
Registration for R/Finance 2015 is now open! The conference will take place on May 29 and 30, at UIC in Chicago. Building on the success of the previous conferences in 2009-2014, we expect more than 250 attendees from around the world. R users from industry, academia, and government will joining 30+ presenters covering all areas of finance with R. We are very excited about the four keynote presentations given by Emanuel Derman, Louis Marascio, Alexander McNeil, and Rishi Narang.
The conference agenda (currently) includes 18 full presentations and 19 shorter "lightning talks". As in previous years, several (optional) pre-conference seminars are offered on Friday morning. There is also an (optional) conference dinner at The Terrace at Trump Hotel. Overlooking the Chicago river and skyline, it is a perfect venue to continue conversations while dining and drinking. Registration information and agenda details can be found on the conference website as they are being finalized.
Registration is also available directly at the registration page. We would to thank our 2015 sponsors for the continued support enabling us to host such an exciting conference: International Center for Futures and Derivatives at UIC Revolution Analytics
MS-Computational Finance and Risk Management at University of Washington Ketchum Trading
OneMarketData
RStudio
SYMMS On behalf of the committee and sponsors, we look forward to seeing you in Chicago! For the program committee:
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich
See you in Chicago in May!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 February 2014

Neil Williams: iMX53 in the home lab

The iMX53 Quick Start Board has been a mainstay of the Debian armhf port and has been part of LAVA until recently. It s a natural choice for seeing how LAVA and Debian can work together more closely. So, in the first steps to getting my home lab running, I set about restoring two of these boards to LAVA support. I will be looking into each of the possible boot methods with these boards, but the way to start is with a LAVA master image on an SD card. With the pause in iMX53 development within LAVA, this raises a couple of issues:
  • u-boot on the iMX53 is old and the Linaro image tools have advanced without making allowances for the unsupported mx53loco type.
    • For LAVA, this means that u-boot on the iMX53 does not understand ext4 and although there are guides on updating it, I wanted to stay with the original, at least at the start.
    • Using ext4 will lead to this error:
      mmc0: error 110 whilst initialising SD card
      
  • iMX53 quick start boards come with a working Ubuntu installation on an SD card but the release is Lucid Lynx 10.04LTS. LAVA has been on Precise Pangolin 12.04LTS for some time.
    • This causes issues with debootstrap which cannot unpack packages from Debian sid due to changes in the compression format inside the .deb files. It is possible to use wheezy and note that lucid, by default, will produce armel. Use --arch=armhf, then apt-get upgrade and apt-get dist-upgrade to testing and then to sid (not in one single step). Remember to mount proc before starting the upgrade.
LAVA master image on iMX53 The LAVA master image is more recent and can fix these problems. See lava-image-creation preparing a master image
$ git clone git://git.linaro.org/lava/lava-master-image-scripts.git
$ cd lava-master-image-scripts

Use patch -p1 to apply this patch (apologies for the line wrapping in the blog):
diff --git a/lava-create-master b/lava-create-master
index 2e1d0e8..a1e6131 100755
--- a/lava-create-master
+++ b/lava-create-master
@@ -366,7 +366,7 @@ make_master()  
         verbose " * Building vanilla image with linaro-media-create..."
         if ! linaro-media-create \
             --dev $LMC_DEV \
-            --rootfs ext4 \
+            --rootfs ext3 \
             --image-file $CACHE_DIR/pristine-images/$dev-vanilla.img \
             --image-size 1G \
             --align-boot-part \
$ sudo ./lava-create-master imx53 Without the patch applied, I simply removed the SD card, mounted the third partition on my main machine and created a tarball of the contents:
$ sudo mount /dev/mmcblk0p3 /mnt/
$ pushd /mnt
$ sudo tar -czf /tmp/imx53-rootfs.tgz ./*
$ popd
$ sudo umount /mnt/
Reformat p3 to ext3 (or ext2 if you prefer)
$ sudo mkfs.ext3 /dev/mmcblk0p3
Put the rootfs back onto the replacement ext3 partition.
$ sudo mount /dev/mmcblk0p3 /mnt/
$ pushd /mnt
$ sudo tar -xzvf /tmp/imx53-rootfs.tgz
$ popd
$ sudo sync
$ sudo umount /mnt/
The bootscript is buggy, so run:
> run loaduimage
> run mmcboot
Then fix bootscript or you won t be able to reboot: Simplest way may be to set u-boot to ignore boot.scr
> setenv script
> saveenv
> reset
Device configuration in LAVA I found screen to be a bit awkward as a serial connection, (use Ctrl-A : quit) to quit. I tried autologin for the lucid image but that was not needed for the master image which uses the linaro overlay. The PDU commands are for my own setup. The daemon (the machine running the pdu daemon scripts), hostname (the hostname or IP of the PDU itself) and the port numbers will change between labs. Again, the blog has changed the line endings.
device_type = mx53loco
hostname = imx53-01
connection_command = telnet hobbes 4002
#login_prompt = lucid-desktop login:
#password_prompt = Password:
#username = lucid
#password = lucid
hard_reset_command = /usr/bin/pduclient --daemon localhost --hostname pdu --command reboot --port 05
power_off_cmd = /usr/bin/pduclient --daemon localhost --hostname pdu --command off --port 05
power_on_cmd = /usr/bin/pduclient --daemon localhost --hostname pdu --command on --port 05
testboot_offset = 3
Critical testboot_offset is currently undocumented. Without it, you will see:
 ** Partition 4 not valid on device 0 **
You can trace this back in a LAVA test log to:
  setenv bootcmd "fatload mmc 0:4 0x70000000 uImage; fatload mmc 0:4 0x72000000 uInitrd; fatload mmc 0:4 0x71ff0000 board.dtb; bootm 0x70000000 0x72000000 0x71ff0000"
The correct line needs to use mmc 0:5 in each case.
This is what testboot_offset modifies the default is 2.
  setenv bootcmd "fatload mmc 0:4 0x70000000 uImage; fatload mmc 0:5 0x72000000 uInitrd; fatload mmc 0:5 0x71ff0000 board.dtb; bootm 0x70000000 0x72000000 0x71ff0000"
Hacking session in LAVA A hacking session in LAVA is a way of getting an SSH login directly into a deployed test image, as root. It s a test image, so look around, trash stuff, build stuff, reboot the board When you are done, use stop-hacking and the test completes, leaving the master image unchanged, ready for the next rootfs to go onto the test image you were using. This will be improved in later blog entries but as a starter and so that you can play with the SATA drive with a recent installation, this is the JSON for a LAVA job which gives you an SSH session inside the LAVA test image on an iMX53. The test image comes from Linaro releases 12.01. I haven t set up a proxy yet, so to save time, I downloaded it once and used a local file:// URL in the testjob. Note the rootfstype parameter without this, LAVA will helpfully format the partition onto which the rootfs is installed to ext4, with predictable results. (This bug is currently being fixed in LAVA so that LAVA uses the same filesystem as the downloaded image. All part of making LAVA less presumptive.) The smoke-tests-basic testdef is gratuitous and can be removed but this particular run won t be quick it took over an hour to get into the hacking session, but future topics here will improve that so the extra few minutes for the smoke-tests is not a big win. Change the value for GATEWAY to something sane for your network, change the value for PUB_KEY to your SSH public key. Change the image location to something which works for your setup.
 
 "health_check": false,
 "job_name": "iMX53-hacking",
 "logging_level": "DEBUG",
 "device_type": "mx53loco",
 "timeout": 900,
 "actions": [
 
 "command": "deploy_linaro_image",
 "parameters": 
 "image": "file:///home/linaro/lava/mx53loco-ubuntu-desktop.img.gz",
 "rootfstype": "ext3"
 
 ,
 
 "command": "lava_test_shell",
 "parameters": 
 "testdef_repos": [
 
 "git-repo": "git://git.linaro.org/qa/test-definitions.git",
 "testdef": "ubuntu/smoke-tests-basic.yaml"
 ,
 
 "git-repo": "http://git.linaro.org/git/lava-team/hacking-session.git",
 "parameters": 
 "GATEWAY": "10.15.0.1",
 "PUB_KEY": "...."
 ,
 "testdef": "hacking-session-debian.yaml"
 
 ],
 "timeout": 18000
 
 ,
 
 "command": "submit_results_on_host",
 "parameters": 
 "server": "http://sylvester.codehelp/RPC2/",
 "stream": "/anonymous/codehelp/"
 
 
 ]
 
(Use http://jsonlint.com to reformat.) I was part of the way through running debootstrap on the SATA drive when ser2net timed out because I hadn t changed the values at that time. Configuring set2net for LAVA ser2net can run on any machine capable of providing the serial connections, including a dedicated device. Decide on a port range to use on the machine running ser2net. If the device has more than one network interface, ensure that ser2net only offers connections on the relevant interface by prefixing the port with the IP address or hostname. telnet is generally the easiest ser2net state to use with LAVA.
#  ::::
#     TCP port
#            Name   or  number of the TCP/IP port to accept con-
#            nections from for this device.  A port number may
#            be of the form [host,]port, such as 127.0.0.1,2000
#            or localhost,2000.  If this is specified, it will
#            only bind to the IP address specified. Otherwise
#            it will bind to all the ports on the machine.
# 10.15.0.1,4000:telnet:600:/dev/ttyUSB0:115200 8DATABITS NONE 1STOPBIT banner
Clear the timeout or long running jobs will fail.
10.15.0.1,4000:telnet:0:/dev/ttyUSB0:115200 8DATABITS NONE 1STOPBIT banner
10.15.0.1,4001:telnet:0:/dev/ttyUSB1:115200 8DATABITS NONE 1STOPBIT banner
I ve tried relatively carefully to preserve the instructions here from my notes. However, if you do try this yourself and there are problems, let me know and I ll update. I m also planning on putting all of the config files, patches, JSON examples and other code into a git repo fairly soon to make things easier.

27 December 2013

Asheesh Laroia: New job (what running Debian means to me)

Five weeks ago, I started a new job (Security Engineer, Eventbrite). I accepted the offer on a Friday evening at about 5:30 PM. That evening, my new boss and I traded emails to help me figure out what kind of computer I'd like. Time was of the essence because my start date was very next day, Tuesday. I wrote about how I value pixel count, and then RAM, and then a speedy disk, and then a speedy CPU. I named a few ThinkPad models that could be good, and with advice from the inimitable danjared, I pointed out that some Dell laptops come pre-installed with Ubuntu (which I could easily swap out for Debian). On Monday, my boss replied. Given the options that the IT department supports, he picked out the best one by my metrics: a MacBook Pro. The IT department would set up the company-mandated full-disk encryption and anti-virus scanning. If I wanted to run Linux, I could set up BootCamp or a virtualization solution. As I read the email, my heart nearly stopped. I just couldn't see myself using a Mac. I thought about it. Does it really matter to me enough to call up my boss and undo an IT request that is already in the works, backpedaling on what I claimed was important to me, opting for brand anti-loyalty to Apple over hardware speed? Yes, I thought to myself. I am willing to just not work there if I have to use a Mac. So I called $BOSS, and I asked, "What can we do to not get me a Mac?" It all worked out fine; I use a ThinkPad X1 Carbon running Debian for work now, and it absolutely does everything I need. It does have a slower CPU, fewer pixels, and less RAM, and I am the only person in the San Francisco engineering office not running Mac OS. But it all works. In the process, I thought it made sense to write up some text to $BOSS. Here is how it goes.
Hi $BOSS,

Thanks for hearing my concerns about having a Mac. It would basically be a fairly serious blow to my self image. It's possible I could rationalize it, but it would take a long time, and I'm not sure it would work.

I don't at all need to start work using the computer I'm going to be using for the weeks afterward. I'm OK with using something temporarily that is whatever is available, Mac or non-Mac; I could happily borrow something out of the equipment closet in the short term if there are plans in the works to replace it with something else that makes me productive in the long term.

For full-disk encryption, there are great solutions for this on Linux.

For anti-virus, it seems Symantec AV is available for Linux <http://www.symantec.com/business/support/index?page=content&id=HOWTO17995>.

It sounds like Apple and possibly Lenovo are the only brands that are available through the IT department, but it is worth mentioning that Dell sells perfectly great laptops with Linux pre-installed, such as the XPS 13. I would perfectly happily use that.

If getting me more RAM is the priority, and the T440s is a bad fit for $COMPANY, then the Lenovo X230 would be a great option, and is noticeably less expensive, and it fits 16GB of RAM.

BootCamp and the like are theoretical possibilities on Macs, but one worry I have is that if there were a configuration issue, it might not be worth me spending work time to have me fix my environment, but instead I would be encouraged for efficiency to use Mac OS, which is well-tested on Apple hardware, and then I would basically hate using my computer, which is a strong emotion, but basically how I would feel.

Another issue (less technical) is that if I took my work machine to the kinds of conferences that I go to, like Debconf, I would find myself in the extremely uncomfortable position of advertising for Apple. I am pretty strongly unexcited about doing that.

Relating to the self-image issue is that it means a lot to me to sort of carry the open source community with me as I do my technical work, even if that technical work is not making more open source software. Feeling part of this world that shares software, and Debian in particular where I have a strong feeling of attachment to the community, even while doing something different, is part of what makes using computers fun for me. So it clashes with that to use Mac OS on my main machine, or to feel like I'm externally indistinguishable from people who don't care about this sort of community.

I am unenthusiastic about making your life harder and looking like a prima donna with my possibly obscure requirements.

I am, however, excited to contribute to $COMPANY!

I hope that helps! Probably nothing you couldn't have guessed in here, but I thought it was worth spelling some of that out. Happy to talk more.

-- Asheesh.

26 December 2013

Ian Campbell: Running ARM Grub on U-boot on Qemu

At the Mini-debconf in Cambridge back in November there was an ARM Sprint (which Hector wrote up as a Bits from ARM porters mail). During this there a brief discussion about using GRUB as a standard bootloader, particularly for ARM server devices. This has the advantage of providing a more "normal" (which in practice means "x86 server-like") as well as flexible solution compared with the existing flash-kernel tool which is often used on ARM. On ARMv7 devices this will more than likely involve chain loading from the U-Boot supplied by the manufacturer. For test and development it would be useful to be able to set up a similar configuration using Qemu. Cross-compilers Although this can be built and run on an ARM system I am using a cross compiler here. I'm using gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux from Linaro, which can be downloaded from the linaro-toolchain-binaries page on Launchpad. (It looks like 2013.10 is the latest available right now, I can't see any reason why that wouldn't be fine). Once the cross-compiler has been downloaded unpack it somewhere, I will refer to the resulting gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux directory as $CROSSROOT. Make sure $CROSSROOT/bin (which contains arm-linux-gnueabihf-gcc etc) is in your $PATH. Qemu I'm using the version packaged in Jessie, which is 1.7.0+dfsg-2. We need both qemu-system-arm for running the final system and qemu-user to run some of the tools. I'd previously tried an older version of qemu (1.6.x?) and had some troubles, although they may have been of my own making... Das U-boot for Qemu First thing to do is to build a suitable u-boot for use in the qemu emulated environment. Since we need to make some configuration changes we need to build from scratch. Start by cloning the upstream git tree:
$ git clone git://git.denx.de/u-boot.git
$ cd u-boot
I am working on top of e03c76c30342 "powerpc/mpc85xx: Update CONFIG_SYS_FSL_TBCLK_DIV for T1040" dated Wed Dec 11 12:49:13 2013 +0530. We are going to use the Versatile Express Cortex-A9 u-boot but first we need to enable some additional configuration options:
  • CONFIG_API -- This enables the u-boot API which Grub uses to access the lowlevel services provided by u-boot. This means that grub doesn't need to contains dozens of platform specific flash, mmc, nand, network, console drivers etc and can be completely platform agnostic.
  • CONFIG_SYS_MMC_MAX_DEVICE -- Setting CONFIG_API needs this.
  • CONFIG_CMD_EXT2 -- Useful for accessing EXT2 formatted filesystems. In this example I use a VFAT /boot for convenience but in a real system we would want to use EXT2 (or even something more modern)).
  • CONFIG_CMD_ECHO -- Just useful.
You can add all these to include/configs/vexpress_common.h:
#define CONFIG_API
#define CONFIG_SYS_MMC_MAX_DEVICE   1
#define CONFIG_CMD_EXT2
#define CONFIG_CMD_ECHO
Or you can apply the patch which I sent upstream:
$ wget -O - http://patchwork.ozlabs.org/patch/304786/raw   git apply --index
$ git commit -m "Additional options for grub-on-uboot"
Finally we can build u-boot:
$ make CROSS_COMPILE=arm-linux-gnueabihf- vexpress_ca9x4_config
$ make CROSS_COMPILE=arm-linux-gnueabihf-
The result is a u-boot binary which we can load with qemu. GRUB for ARM Next we can build grub. Start by cloning the upstream git tree:
$ git clone git://git.sv.gnu.org/grub.git
$ cd grub
By default grub is built for systems which have RAM at address 0x00000000. However the Versatile Express platform which we are targeting has RAM starting from 0x60000000 so we need to make a couple of modifications. First in grub-core/Makefile.core.def we need to change arm_uboot_ldflags, from:
-Wl,-Ttext=0x08000000
to
-Wl,-Ttext=0x68000000
and second we need make a similar change to include/grub/offsets.h changing GRUB_KERNEL_ARM_UBOOT_LINK_ADDR from 0x08000000 to 0x68000000. Now we are ready to build grub:
$ ./autogen.sh
$ ./configure --host arm-linux-gnueabihf
$ make
Now we need to build the final grub "kernel" image, normally this would be taken care of by grub-install but because we are cross building grub we cannot use this and have to use grub-mkimage directly. However the version we have just built is for the ARM target and not for host we are building things on. I've not yet figured out how to build grub for ARM while building the tools for the host system (I'm sure it is possible somehow...). Luckily we can use qemu to run the ARM binary:
$ cat load.cfg
set prefix=(hd0)
$ qemu-arm -r 3.11 -L $CROSSROOT/arm-linux-gnueabihf/libc \
    ./grub-mkimage -c load.cfg -O arm-uboot -o core.img -d grub-core/ \
    fat ext2 probe terminal scsi ls linux elf msdospart normal help echo
Here we create load.cfg which is the setup script which will be built into the grub kernel, our version just sets the root device so that grub can find the rest of its configuration. Then we use qemu-arm-static to invoke grub-mkimage. The "-r 3.11" option tells qemu to pretend to be a 3.11 kernel (which is required by the libc used by our cross compiler, without this you will get a fatal: kernel too old message) and "-L $CROSSROOT/..." tells it where to find the basic libraries, such as the dynamic linker (luckily grub-mkimage doesn't need much in the way of libraries so we don't need a full cross library environment. The grub-mkimage command passes in the load.cfg and requests an output kernel targeting arm-uboot, core.img is the output file and the modules are in grub-core (because we didn't actually install grub in the target system, normally these would be found in /boot/grub). Lastly we pass in a list of default modules to build into the kernel, including filesystem drivers (fat, ext2), disk drivers (scsi), partition handling (msdos), loaders (linux, elf), the menu system (normal) and various other bits and bobs. So after all the we now have our grub kernel in core.img. Putting it all together Before we can launch qemu we need to create various disk images. Firstly we need some images for the 2 64M flash devices:
$ dd if=/dev/zero of=pflash0.img bs=1M count=64
$ dd if=/dev/zero of=pflash1.img bs=1M count=64
We will initialise these later from the u-boot command line. Secondly we need an image for the root filesystem on an MMC device. I'm using a FAT formatted image here simply for the convenience of using mtools to update the images during development.
$ dd if=/dev/zero of=mmc.img bs=1M count=16
$ /sbin/mkfs.vfat mmc.img
Thirdly we need a kernel, device tree and grub configuration on our root filesystem. For the first two I extracted them from the standard armmp kernel flavour package. I used the backports.org version 3.11-0.bpo.2-armmp version and extracted /boot/vmlinuz-3.11-0.bpo.2-armmp as vmlinuz and /usr/lib/linux-image-3.11-0.bpo.2-armmp/vexpress-v2p-ca9.dtb as dtb. Then I hand coded a simple grub.cfg:
menuentry 'Linux'  
        echo "Loading vmlinuz"
        set root='hd0'
        linux /vmlinuz console=ttyAMA0 ro debug
        devicetree /dtb
 
In a real system the kernel and dtb would be provided by the kernel packages and grub.cfg would be generated by update-grub. Now that we have all the bits we need copy them into the root of mmc.img. Since we are using a FAT formatted image we can use mcopy from the mtools package.
$ mcopy -v -o -n -i mmc.img core.img dtb vmlinuz grub.cfg ::
Finally after all that we can run qemu passing it our u-boot binary and the mmc and flash images and requesting a Cortex-A9 based Versatile Express system with 1GB of RAM:
$ qemu-system-arm -M vexpress-a9 -kernel u-boot -m 1024m -sd mmc.img \
    -nographic -pflash pflash0.img -pflash pflash1.img
Then at the VExpress# prompt we can configure the default bootcmd to load grub and save the environment to the flash images. The backslash escapes (\$ and \;) should be included as written here so that e.g. the variables are only evaluated when bootcmd is evaluated and not immediately when setting bootcmd and the bootm is set as part of bootcmd instead of executed immediately:
VExpress# setenv bootcmd fatload mmc 0:0 \$ loadaddr  core.img \; bootm \$ loadaddr 
VExpress# saveenv
Now whenever we boot the system it will automatically load boot grub from the mmc and launch it. Grub in turn will load the Linux binary and DTB and launch those. I haven't actually configure Linux with a root filesystem here so it will eventually panic after failing to find root. Future work The most pressing issue is the hard coded load address built in to the grub kernel image. This is something which needs to be discussed with the upstream grub maintainers as well as the Debian package maintainers. Now that the ARM packages have hit Debian (in experimental in the 2.02~beta2-1 package) I also plan to start looking at debian-installer integration as well as updating flash-kernel to setup the chain load of grub instead of loading a kernel directly.

12 June 2013

Marc 'Zugschlus' Haber: How to amd64 an i386 Debian installation with multiarch

Migrating a Debian installation between architectures has always been difficult. The recommended way to crossgrade an i386 Debian to amd64 Debian was to reinstall the system, move over data and configuration. For the more brave, in-place crossgrades usually involved chroots, rescue CDs, a lot of ar p tar xf - data.tar.gz and luck. I have never been brave when it comes to system administration, have done a lot of architecture migrations with reinstallation, and have always taken the opportunity to clear out the contamination that accumulates itself when a system is running for a long time. I would even recommend doing this to most people even now. However, I have a few very ugly systems in place that are still on i386 because I didn t dare going the reinstallation path. Doing in-place crossgrades has become a lot easier since wheezy s release, since once now can have both i386 and amd64 libraries installed in parallel, which allows to replace foo:i386 with foo:amd64 without influencing the other parts of the system. The process is still full of pitfalls:
  • apt removes a package before it reinstalls its new counterpart. This results in apt calling dpkg to remove dpkg, and then calling dpkg again to install dpkg. Guess which operation fails and the state of the system after this failure. Same applies to coreutils, which leaves the system without rm, which in turn dpkg of either architecture doesn t like. Using apt-get --download-only install to resolve dependencies and downloading the debs, followed by a traditional dpkg --install solves this issue since multiarch dpkg will replace a package with another one without deinstalling the first one first.
  • At least for the process, you need a kernel that can run both 32bit and 64bit binaries for the i386 architecture. AFAIR, setting CONFIG_64BIT, CONFIG_X86_64 and CONFIG_IA32_EMULATION in the kernel configuration takes care of this.
  • During the process, apt wants to remove the better part of your system. It is important to not let it do this, as it wants to deinstall essential packages as well.
  • It looks like the process is not always exactly reproducible. During the first tries, I found myself without an initrd at all, with an initrd that lacked the ext[234].ko kernel modules, without working e2fstools and in a number of other undesireable states of the system.
I have only tried this yet with a freshly installed minimal wheezy server system. I do not, however, expect surprises when it comes to using this process with real life systems. I will document other pitfalls I have fallen into here at a later time. My minimal wheezy system was running in a KVM VM with its virtual disk as a LVM LV in the host system. I took a snapshot before beginning and used lvconvert --merge numerous time to return my LV to the original state. Be aware that lvconvert --merge removes the snapshot after merging it, so you ll need to re-create the snapshot before trying again. During the process, I discussed things with Paul Tagliamonte, who has done this before, but on a live system and with a slightly more invasive approach. He has blogged about this. Thank you very much, your hints were very helpful. Here is a commented typescript of what I have done. Be warned: This was a lab setting with a rather minimal system. If you re going to try this with a production system, have a backup, or, if you re on snapshottable infrastructure, take snapshots and be prepared to roll back if anything goes wrong. First let s see what we have and save a list of installed packages.
mh@swivel:~$ ssh wheezyarch.zugschlus.de
$ uname -a
Linux wheezyarch 3.9.4-zgsrv2008064 #2 SMP PREEMPT Wed Jun 5 12:57:51 UTC 2013 x86_64 GNU/Linux
$ dpkg --print-architecture
i386
$ dpkg --print-foreign-architectures
$ dpkg --list   grep i386   wc -l
175
$ dpkg --list   grep amd64   wc -l
0
$ sudo dpkg --get-selections   grep install   awk  print $1    sed  s/:.*//;s/$/:amd64
install/;  > get-selections.pre
So we have an i386 system running an x86_64 kernel. Next, we add amd64 as an additional architecture which allows us to install amd64 packages. We download the amd64 packages files. Note that dpkg still considers this an i386 system with amd64 as a foreign arch.
$ sudo dpkg --add-architecture amd64
$ sudo apt-get update
Get:1 http://security.debian.org wheezy/updates Release.gpg [836 B]
( )
Fetched 6.155 kB in 4s (1.400 kB/s)
Reading package lists... Done
$ dpkg --print-architecture
i386
$ dpkg --print-foreign-architectures
amd64
$ dpkg --list   grep i386   wc -l
175
$ dpkg --list   grep amd64   wc -l
0
Next, download everything that is necessary to migrate dpkg and apt from i386 and amd64. We cannot directly apt things as the remove-install cycle used by apt will leave us without working dpkg. One could work around this by un-aring the dpkg.deb and directly forcing the amd64 files upon the system, but I think this approach is much cleaner. apt-get download will not resolve dependices while apt-get --download install does. After the download, dpkg --install the packages. dpkg is not particularly smart about installation order, so Pre-Depends may fail or not. If they fail, dpkg will give a list of deb files that were affected, which is an easy help to finish their installation. One could also choose the easy way (which I did here) and repeat the dpkg --install *.deb command. Since the Pre-Depends will already be installed, the second install run will succeed.
$ sudo apt-get --download-only install dpkg:amd64 apt:amd64
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  gcc-4.7-base:amd64 libapt-pkg4.12:amd64 libbz2-1.0:amd64 libc6:amd64
  libgcc1:amd64 liblzma5:amd64 libselinux1:amd64 libstdc++6:amd64
  zlib1g:amd64
Suggested packages:
  aptitude:amd64 synaptic:amd64 wajig:amd64 dpkg-dev:amd64 apt-doc:amd64
  python-apt:amd64 glibc-doc:amd64 locales:amd64
The following packages will be REMOVED:
  apt cron-apt dpkg
The following NEW packages will be installed:
  apt:amd64 dpkg:amd64 gcc-4.7-base:amd64 libapt-pkg4.12:amd64
  libbz2-1.0:amd64 libc6:amd64 libgcc1:amd64 liblzma5:amd64 libselinux1:amd64
  libstdc++6:amd64 zlib1g:amd64
0 upgraded, 11 newly installed, 3 to remove and 0 not upgraded.
Need to get 10,0 MB of archives.
After this operation, 14,9 MB of additional disk space will be used.
Do you want to continue [Y/n]? 
Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main gcc-4.7-base amd64 4.7.2-5 [144 kB]
( )
Fetched 10,0 MB in 2s (3.535 kB/s)
Download complete and in download only mode
$ sudo dpkg --install /var/cache/apt/archives/*.deb
(Reading database ... 16586 files and directories currently installed.)
Preparing to replace apt 0.9.7.8 (using .../archives/apt_0.9.7.8_amd64.deb) ...
Unpacking replacement apt ...
dpkg: regarding .../dpkg_1.16.10_amd64.deb containing dpkg, pre-dependency problem:
 dpkg pre-depends on libbz2-1.0
dpkg: error processing /var/cache/apt/archives/dpkg_1.16.10_amd64.deb (--install):
 pre-dependency problem - not installing dpkg
Selecting previously unselected package gcc-4.7-base:amd64.
( )
Errors were encountered while processing:
 /var/cache/apt/archives/dpkg_1.16.10_amd64.deb
$ sudo dpkg --install /var/cache/apt/archives/*.deb
(Reading database ... 16905 files and directories currently installed.)
Preparing to replace apt 0.9.7.8 (using .../archives/apt_0.9.7.8_amd64.deb) ...
Unpacking replacement apt ...
( )
$
From dpkg s point of view, the system is now already amd64, with i386 being a foreign arch. However, the majory of packages is still i386.
$ dpkg --print-architecture
amd64
$ dpkg --print-foreign-architectures
i386
$ dpkg --list   grep i386   wc -l
173
$ dpkg --list   grep amd64   wc -l
11
$ 
The apt resolver is currently in a very bad broken state since the system is missing most essential packages in its native architecture. This is not easily solved by apt-get -f install, as this would zap most of the existing system . We let apt-get --download-only -f install download the packages that it wants to replace with its amd64 counterparts and their dependencies, and, again, use dpkg --install *.deb to install them.
$ sudo apt-get --download-only -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  cpio gettext-base:i386 klibc-utils libapt-inst1.5 libasprintf0c2:i386
  libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386 libklibc
  libreadline5:i386
Use  apt-get autoremove  to remove them.
The following extra packages will be installed:
  acpid aide bsd-mailx cpio cron curl daemon dctrl-tools exim4-base
  exim4-daemon-light initscripts insserv kbd klibc-utils less libbsd0
  libcomerr2 libcurl3 libdb5.1 libexpat1 libgcrypt11 libgdbm3 libgnutls26
  libgpg-error0 libgssapi-krb5-2 libidn11 libk5crypto3 libkeyutils1 libklibc
  libkmod2 libkrb5-3 libkrb5support0 libldap-2.4-2 liblockfile1 libncursesw5
  libnewt0.52 libp11-kit0 libpam-modules libpam0g libpcre3 libpopt0
  libreadline6 librtmp0 libsasl2-2 libslang2 libsqlite3-0 libssh2-1
  libssl1.0.0 libtasn1-3 libterm-readkey-perl libtinfo5 openssl perl
  perl-base python2.7 python2.7-minimal sysvinit-utils whiptail
Suggested packages:
  libarchive1 anacron logrotate checksecurity debtags mail-reader eximon4
  exim4-doc-html exim4-doc-info spf-tools-perl swaks bootchart2 rng-tools
  krb5-doc krb5-user libpam-doc perl-doc make python2.7-doc binutils
  binfmt-support bootlogd sash
Recommended packages:
  exim4 postfix mail-transport-agent psmisc mailx ca-certificates
  krb5-locales libgpm2 libfribidi0 libsasl2-modules libpng12-0
The following packages will be REMOVED:
  acpid:i386 aide:i386 aptitude:i386 bsd-mailx:i386 cpio:i386 cron:i386
  curl:i386 daemon:i386 dctrl-tools:i386 debsecan dmsetup:i386 e2fsprogs:i386
  exim4-base:i386 exim4-daemon-light:i386 git:i386 grub-common:i386
  grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386
  initramfs-tools initscripts:i386 insserv:i386 ippl:i386 jed:i386 kbd:i386
  klibc-utils:i386 less:i386 libdevmapper-event1.02.1:i386
  libdevmapper1.02.1:i386 libklibc:i386 libnewt0.52:i386
  libparted0debian1:i386 libterm-readkey-perl:i386 lsof:i386 lvm2:i386
  molly-guard ntp:i386 openssh-server:i386 openssl:i386 parted:i386 perl:i386
  perl-base:i386 procps:i386 python-apt:i386 python2.7:i386
  python2.7-minimal:i386 rsyslog:i386 sysvinit:i386 sysvinit-utils:i386
  udev:i386 util-linux:i386 whiptail:i386
The following NEW packages will be installed:
  acpid aide bsd-mailx cpio cron curl daemon dctrl-tools exim4-base
  exim4-daemon-light initscripts insserv kbd klibc-utils less libbsd0
  libcomerr2 libcurl3 libdb5.1 libexpat1 libgcrypt11 libgdbm3 libgnutls26
  libgpg-error0 libgssapi-krb5-2 libidn11 libk5crypto3 libkeyutils1 libklibc
  libkmod2 libkrb5-3 libkrb5support0 libldap-2.4-2 liblockfile1 libncursesw5
  libnewt0.52 libp11-kit0 libpam-modules libpam0g libpcre3 libpopt0
  libreadline6 librtmp0 libsasl2-2 libslang2 libsqlite3-0 libssh2-1
  libssl1.0.0 libtasn1-3 libterm-readkey-perl libtinfo5 openssl perl
  perl-base python2.7 python2.7-minimal sysvinit-utils whiptail
0 upgraded, 58 newly installed, 53 to remove and 0 not upgraded.
Need to get 23,4 MB of archives.
After this operation, 17,5 MB disk space will be freed.
Do you want to continue [Y/n]? Y
Get:1 http://security.debian.org/debian-security/ wheezy/updates/main libgnutls26 amd64 2.12.20-7 [619 kB]
( )
Fetched 23,4 MB in 11s (2.038 kB/s)                                           
Download complete and in download only mode
$ sudo dpkg --install /var/cache/apt/archives/*.deb
(Reading database ... 16905 files and directories currently installed.)
Preparing to replace acpid 1:2.0.16-1+deb7u1 (using .../acpid_1%3a2.0.16-1+deb7u1_amd64.deb) ...
( )
Selecting previously unselected package libpam-modules:amd64.
dpkg: regarding .../libpam-modules_1.1.3-7.1_amd64.deb containing libpam-modules:amd64, pre-dependency problem:
 libpam-modules pre-depends on libdb5.1
  libdb5.1:amd64 is unpacked, but has never been configured.
dpkg: error processing /var/cache/apt/archives/libpam-modules_1.1.3-7.1_amd64.deb (--install):
 pre-dependency problem - not installing libpam-modules:amd64
Selecting previously unselected package libpcre3:amd64.
Unpacking libpcre3:amd64 (from .../libpcre3_1%3a8.30-5_amd64.deb) ...
( )
Errors were encountered while processing:
 /var/cache/apt/archives/libpam-modules_1.1.3-7.1_amd64.deb
$ sudo dpkg --install /var/cache/apt/archives/libpam-modules_1.1.3-7.1_amd64.deb
(Reading database ... 17007 files and directories currently installed.)
Unpacking libpam-modules:amd64 (from .../libpam-modules_1.1.3-7.1_amd64.deb) ...
Setting up libpam-modules:amd64 (1.1.3-7.1) ...
$ sudo apt-get clean
$ dpkg --print-architecture
amd64
$ dpkg --print-foreign-architectures
i386
$ dpkg --list   grep i386   wc -l
151
$ dpkg --list   grep amd64   wc -l
66
$ 
The concluding apt-get clean prevents packages from being reinstalled again in the next dpkg --install *.deb run. Now, the apt resolver is kind of fixed, but apt will still want to remove most of the system. I haven t found a way to get out of this mess short of allowing apt to de-install those packages and to manually install their amd64 counterparts later. First, we see what apt wants to do, and paste the The following packages will be removed part of apt-get s output into a sed expression which fixes the architecture strings for us so that we can use the resulting file as input for the following installation procedure. After that, we let apt-get do what itself considers a bad thing and requires us to type a consent sentence. The key is that this deinstallation of essential packages breaks the system, but not apt and dpkg, which allows us to un-break the system afterwards.
$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  gettext-base:i386 libapt-inst1.5 libasprintf0c2:i386 libcurl3-gnutls:i386
  libfreetype6:i386 libfuse2:i386 libreadline5:i386
Use  apt-get autoremove  to remove them.
The following extra packages will be installed:
  sysvinit-utils:i386
Suggested packages:
  bootlogd:i386 sash:i386
The following packages will be REMOVED:
  adduser aide-common aptitude:i386 bsd-mailx console-common console-data
  console-log cron curl debian-goodies debsecan dmsetup:i386 e2fsprogs:i386
  exim4-base exim4-config exim4-daemon-light git:i386 grub-common:i386
  grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386
  initramfs-tools initscripts ippl:i386 jed:i386 libcurl3
  libdevmapper-event1.02.1:i386 libdevmapper1.02.1:i386 liberror-perl
  libfile-find-rule-perl libnumber-compare-perl libparted0debian1:i386
  libswitch-perl libterm-readkey-perl libterm-readline-perl-perl
  libtext-glob-perl libtimedate-perl locales lsof:i386 lvm2:i386 molly-guard
  ntp:i386 openssh-client:i386 openssh-server:i386 openssl openssl-blacklist
  openssl-blacklist-extra parted:i386 perl perl-modules procps:i386
  python-apt:i386 rsyslog:i386 stow sysv-rc sysvinit:i386 sysvinit-utils
  tzdata ucf udev:i386 util-linux:i386
The following NEW packages will be installed:
  sysvinit-utils:i386
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
  e2fsprogs:i386 util-linux:i386 (due to e2fsprogs:i386) sysvinit:i386
  sysvinit-utils tzdata (due to util-linux:i386)
0 upgraded, 1 newly installed, 62 to remove and 0 not upgraded.
Need to get 97,1 kB of archives.
After this operation, 131 MB disk space will be freed.
You are about to do something potentially harmful.
To continue type in the phrase  Yes, do as I say!  ? (abort with Ctrl-C)
$ sed  s/:i386//g  > removedpkg (paste  The following packages will be REMOVED:  stanza here and
type Ctrl-D afterwards)
  adduser aide-common aptitude:i386 bsd-mailx console-common console-data
  console-log cron curl debian-goodies debsecan dmsetup:i386 e2fsprogs:i386
  exim4-base exim4-config exim4-daemon-light git:i386 grub-common:i386
  grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386
  initramfs-tools initscripts ippl:i386 jed:i386 libcurl3
  libdevmapper-event1.02.1:i386 libdevmapper1.02.1:i386 liberror-perl
  libfile-find-rule-perl libnumber-compare-perl libparted0debian1:i386
  libswitch-perl libterm-readkey-perl libterm-readline-perl-perl
  libtext-glob-perl libtimedate-perl locales lsof:i386 lvm2:i386 molly-guard
  ntp:i386 openssh-client:i386 openssh-server:i386 openssl openssl-blacklist
  openssl-blacklist-extra parted:i386 perl perl-modules procps:i386
  python-apt:i386 rsyslog:i386 stow sysv-rc sysvinit:i386 sysvinit-utils
  tzdata ucf udev:i386 util-linux:i386
$ cat removedpkg 
  adduser aide-common aptitude bsd-mailx console-common console-data
  console-log cron curl debian-goodies debsecan dmsetup e2fsprogs
  exim4-base exim4-config exim4-daemon-light git grub-common
  grub-pc grub-pc-bin grub2-common ifupdown
  initramfs-tools initscripts ippl jed libcurl3
  libdevmapper-event1.02.1 libdevmapper1.02.1 liberror-perl
  libfile-find-rule-perl libnumber-compare-perl libparted0debian1
  libswitch-perl libterm-readkey-perl libterm-readline-perl-perl
  libtext-glob-perl libtimedate-perl locales lsof lvm2 molly-guard
  ntp openssh-client openssh-server openssl openssl-blacklist
  openssl-blacklist-extra parted perl perl-modules procps
  python-apt rsyslog stow sysv-rc sysvinit sysvinit-utils
  tzdata ucf udev util-linux
$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  gettext-base:i386 libapt-inst1.5 libasprintf0c2:i386 libcurl3-gnutls:i386
  libfreetype6:i386 libfuse2:i386 libreadline5:i386
Use  apt-get autoremove  to remove them.
The following extra packages will be installed:
  sysvinit-utils:i386
Suggested packages:
  bootlogd:i386 sash:i386
The following packages will be REMOVED:
  adduser aide-common aptitude:i386 bsd-mailx console-common console-data
  console-log cron curl debian-goodies debsecan dmsetup:i386 e2fsprogs:i386
  exim4-base exim4-config exim4-daemon-light git:i386 grub-common:i386
  grub-pc:i386 grub-pc-bin:i386 grub2-common:i386 ifupdown:i386
  initramfs-tools initscripts ippl:i386 jed:i386 libcurl3
  libdevmapper-event1.02.1:i386 libdevmapper1.02.1:i386 liberror-perl
  libfile-find-rule-perl libnumber-compare-perl libparted0debian1:i386
  libswitch-perl libterm-readkey-perl libterm-readline-perl-perl
  libtext-glob-perl libtimedate-perl locales lsof:i386 lvm2:i386 molly-guard
  ntp:i386 openssh-client:i386 openssh-server:i386 openssl openssl-blacklist
  openssl-blacklist-extra parted:i386 perl perl-modules procps:i386
  python-apt:i386 rsyslog:i386 stow sysv-rc sysvinit:i386 sysvinit-utils
  tzdata ucf udev:i386 util-linux:i386
The following NEW packages will be installed:
  sysvinit-utils:i386
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
  e2fsprogs:i386 util-linux:i386 (due to e2fsprogs:i386) sysvinit:i386
  sysvinit-utils tzdata (due to util-linux:i386)
0 upgraded, 1 newly installed, 62 to remove and 0 not upgraded.
Need to get 97,1 kB of archives.
After this operation, 131 MB disk space will be freed.
You are about to do something potentially harmful.
To continue type in the phrase  Yes, do as I say! 
 ?] Yes, do as I say! (actually type this!)
Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main sysvinit-utils i386 2.88dsf-41 [97,1 kB]
Fetched 97,1 kB in 0s (687 kB/s)    
(Reading database ... 17050 files and directories currently installed.)
Removing aide-common ...
( )
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
Removing e2fsprogs ...
Removing sysv-rc ...
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
Removing sysvinit ...
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
Removing sysvinit-utils ...
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
Removing ucf ...
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
Removing libdevmapper1.02.1:i386 ...
Removing libswitch-perl ...
Removing perl ...
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
Removing dmsetup ...
Removing perl-modules ...
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
Removing util-linux ...
Removing tzdata ...
Processing triggers for mime-support ...
Selecting previously unselected package sysvinit-utils.
(Reading database ... 9802 files and directories currently installed.)
Unpacking sysvinit-utils (from .../sysvinit-utils_2.88dsf-41_i386.deb) ...
Setting up sysvinit-utils (2.88dsf-41) ...
$ sudo apt-get install $(cat removedpkg)
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386
  libreadline5:i386
Use  apt-get autoremove  to remove them.
The following extra packages will be installed:
  e2fslibs gettext-base libapt-inst1.5 libasprintf0c2 libattr1 libblkid1
  libboost-iostreams1.49.0 libcap2 libcurl3-gnutls libcwidget3 libedit2
  libept1.4.12 libfreetype6 libfuse2 libgpm2 libncurses5 libopts25 libprocps0
  libreadline5 libsepol1 libsigc++-2.0-0c2a libss2 libudev0 libuuid1 libwrap0
  libxapian22 logrotate
Suggested packages:
  liblocale-gettext-perl tasksel debtags unicode-data anacron checksecurity
  popularity-contest xdg-utils zenity gpart e2fsck-static mail-reader eximon4
  exim4-doc-html exim4-doc-info spf-tools-perl swaks git-daemon-run
  git-daemon-sysvinit git-doc git-el git-arch git-cvs git-svn git-email
  git-gui gitk gitweb multiboot-doc grub-emu xorriso desktop-base
  isc-dhcp-client dhcp-client ppp rdnssd net-tools gpm libcwidget-dev fuse
  libparted0-dev libparted0-i18n xapian-tools ntp-doc ssh-askpass libpam-ssh
  keychain monkeysphere rssh ufw parted-doc perl-doc make libpod-plainer-perl
  python-apt-dbg python-gtk2 python-vte python-apt-doc rsyslog-mysql
  rsyslog-pgsql rsyslog-doc rsyslog-gnutls rsyslog-gssapi rsyslog-relp
  doc-base sysv-rc-conf bum bootlogd sash util-linux-locales dosfstools
Recommended packages:
  aptitude-doc-en aptitude-doc apt-xapian-index libparse-debianchangelog-perl
  exim4 postfix mail-transport-agent psmisc mailx patch rsync ssh-client
  os-prober busybox busybox-initramfs busybox-static ca-certificates
  uuid-runtime xauth ncurses-term iso-codes usbutils
The following packages will be REMOVED:
  gettext-base:i386 libboost-iostreams1.49.0:i386 libcwidget3:i386
  libept1.4.12:i386 libopts25:i386 libxapian22:i386 logrotate:i386
  sysvinit-utils:i386
The following NEW packages will be installed:
  adduser aide-common aptitude bsd-mailx console-common console-data
  console-log cron curl debian-goodies debsecan dmsetup e2fslibs e2fsprogs
  exim4-base exim4-config exim4-daemon-light gettext-base git grub-common
  grub-pc grub-pc-bin grub2-common ifupdown initramfs-tools initscripts ippl
  jed libapt-inst1.5 libasprintf0c2 libattr1 libblkid1
  libboost-iostreams1.49.0 libcap2 libcurl3 libcurl3-gnutls libcwidget3
  libdevmapper-event1.02.1 libdevmapper1.02.1 libedit2 libept1.4.12
  liberror-perl libfile-find-rule-perl libfreetype6 libfuse2 libgpm2
  libncurses5 libnumber-compare-perl libopts25 libparted0debian1 libprocps0
  libreadline5 libsepol1 libsigc++-2.0-0c2a libss2 libswitch-perl
  libterm-readkey-perl libterm-readline-perl-perl libtext-glob-perl
  libtimedate-perl libudev0 libuuid1 libwrap0 libxapian22 locales logrotate
  lsof lvm2 molly-guard ntp openssh-client openssh-server openssl
  openssl-blacklist openssl-blacklist-extra parted perl perl-modules procps
  python-apt rsyslog stow sysv-rc sysvinit sysvinit-utils tzdata ucf udev
  util-linux
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
  sysvinit-utils:i386
0 upgraded, 89 newly installed, 8 to remove and 0 not upgraded.
Need to get 55.3 MB of archives.
After this operation, 134 MB of additional disk space will be used.
You are about to do something potentially harmful.
To continue type in the phrase  Yes, do as I say! 
 ?] Yes, do as I say! (actually type this!)
Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main e2fslibs amd64 1.42.5-1.1 [197 kB]
( )
Fetched 55.3 MB in 20s (2648 kB/s)                                            
perl: warning: Setting locale failed.
( )
This process may re-ask some debconf questions that you have already answered during initial system setup. I find it comforting to actually see that my answers were preserved, but you can always export DEBIAN_FRONTEND=noninteractive if you want installation to be silent. If you know how this step can be done more elegantly, please comment. We now, finally, have the resolver in a consistent state.
$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386
  libreadline5:i386
Use  apt-get autoremove  to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
$ dpkg --print-architecture
amd64
$ dpkg --print-foreign-architectures
i386
$ dpkg --list   grep i386   wc -l
121
$ dpkg --list   grep amd64   wc -l
119
$ 
Now we can force-crossgrade all i386 packages that are still left on the system. Again, we use the apt-get --download install, dpkg --install stunt as coreutils gets crossgraded at this step and dpkg is less than happy if it finds itself without /bin/rm. Remember, with --download, nothing is actually removed during this step even if apt-get claims to remove things. Apt will also complain that many of these packages are already installed. Those are libs that were pulled in previously. The first dpkg --install run in this step will complain about gazillions of dependency problems, which will all be resolved in concluding apt-get -f install after the second dpkg --install run, this time a manual one.
$ sudo apt-get --download-only install $(dpkg --list   awk  if( $1 ==  ii  && $4 ==
 i386  )   print $2     sed  s/:i386// )
Reading package lists... Done
Building dependency tree       
Reading state information... Done
( )
The following packages were automatically installed and are no longer required:
  libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386
  libreadline5:i386
Use  apt-get autoremove  to remove them.
Suggested packages:
  powermgmt-base bash-doc bzip2-doc diffutils-doc wdiff mlocate locate
  gnupg-doc xloadimage imagemagick eog libpcsclite1 iproute-doc resolvconf
  avahi-autoipd nfs-common ncompress indent
Recommended packages:
  bsdmainutils gnupg-curl libatm1
The following packages will be REMOVED:
  anacron:i386 apt-utils:i386 base-files:i386 base-passwd:i386 bash:i386
( )
The following NEW packages will be installed:
  anacron apt-utils base-files base-passwd bash bsdutils busybox bzip2
( )
0 upgraded, 56 newly installed, 49 to remove and 0 not upgraded.
Need to get 24,1 MB of archives.
After this operation, 2.030 kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://debian.debian.zugschlus.de/debian/ wheezy/main kmod amd64 9-3 [60,5 kB]
( )
Fetched 24,1 MB in 11s (2.120 kB/s)                                           
Download complete and in download only mode
$ sudo dpkg --install /var/cache/apt/archives/*.deb
(Reading database ... 17098 files and directories currently installed.)
Preparing to replace adduser 3.113+nmu3 (using .../adduser_3.113+nmu3_all.deb) ...
Unpacking replacement adduser ...
( )
Errors were encountered while processing:
 /var/cache/apt/archives/bash_4.2+dfsg-0.1_amd64.deb
 /var/cache/apt/archives/coreutils_8.13-3.5_amd64.deb
 initscripts
 lvm2
 procps
 rsyslog
 sysvinit
 sysv-rc
 udev
 util-linux
 aide-common
 dmsetup
 e2fsprogs
 ifupdown
 initramfs-tools
 libdevmapper1.02.1:amd64
 libdevmapper-event1.02.1:amd64
 libparted0debian1:amd64
 molly-guard
 openssh-server
 parted
 grub-common
 grub-pc
 grub-pc-bin
 grub2-common
$ sudo dpkg --install /var/cache/apt/archives/bash_4.2+dfsg-0.1_amd64.deb
/var/cache/apt/archives/coreutils_8.13-3.5_amd64.deb
(Reading database ... 17129 files and directories currently installed.)
Preparing to replace bash 4.2+dfsg-0.1 (using .../bash_4.2+dfsg-0.1_amd64.deb) ...
Unpacking replacement bash ...
Preparing to replace coreutils 8.13-3.5 (using .../coreutils_8.13-3.5_amd64.deb) ...
Unpacking replacement coreutils ...
Setting up bash (4.2+dfsg-0.1) ...
update-alternatives: using /usr/share/man/man7/bash-builtins.7.gz to provide /usr/share/man/man7/builtins.7.gz
(builtins.7.gz) in auto mode
Setting up coreutils (8.13-3.5) ...
$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386
  libreadline5:i386
Use  apt-get autoremove  to remove them.
The following extra packages will be installed:
  sysvinit-utils
Suggested packages:
  bootlogd sash
The following packages will be REMOVED:
  libslang2-modules:i386 sysvinit-utils:i386
The following NEW packages will be installed:
  sysvinit-utils
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
  sysvinit-utils:i386
0 upgraded, 1 newly installed, 2 to remove and 0 not upgraded.
23 not fully installed or removed.
Need to get 0 B/99,5 kB of archives.
After this operation, 328 kB disk space will be freed.
You are about to do something potentially harmful.
To continue type in the phrase  Yes, do as I say! 
 ?] Yes, do as I say! (actually type this!)
dpkg: warning: overriding problem because --force enabled:
 This is an essential package - it should not be removed.
(Reading database ... 17129 files and directories currently installed.)
Removing sysvinit-utils ...
Selecting previously unselected package sysvinit-utils.
(Reading database ... 17105 files and directories currently installed.)
Unpacking sysvinit-utils (from .../sysvinit-utils_2.88dsf-41_amd64.deb) ...
Setting up sysvinit-utils (2.88dsf-41) ...
Setting up sysv-rc (2.88dsf-41) ...
Setting up initscripts (2.88dsf-41) ...
Setting up util-linux (2.20.1-5.3) ...
Setting up e2fsprogs (1.42.5-1.1) ...
Setting up sysvinit (2.88dsf-41) ...
sysvinit: restarting... done.
(Reading database ... 17129 files and directories currently installed.)
Removing libslang2-modules:i386 ...
Setting up ifupdown (0.7.8) ...
Setting up procps (1:3.3.3-3) ...
[ ok ] Setting kernel variables ... /etc/sysctl.conf...done.
Setting up udev (175-7.2) ...
[ ok ] Stopping the hotplug events dispatcher: udevd.
[ ok ] Starting the hotplug events dispatcher: udevd.
update-initramfs: deferring update (trigger activated)
Setting up rsyslog (5.8.11-3) ...
[ ok ] Stopping enhanced syslogd: rsyslogd.
[ ok ] Starting enhanced syslogd: rsyslogd.
Setting up aide-common (0.15.1-8) ...
Setting up initramfs-tools (0.109.1) ...
update-initramfs: deferring update (trigger activated)
Setting up openssh-server (1:6.0p1-4) ...
[ ok ] Restarting OpenBSD Secure Shell server: sshd.
Setting up molly-guard (0.4.5-1) ...
Setting up libdevmapper1.02.1:amd64 (2:1.02.74-7) ...
Setting up libdevmapper-event1.02.1:amd64 (2:1.02.74-7) ...
Setting up libparted0debian1:amd64 (2.3-12) ...
Setting up grub-common (1.99-27+deb7u1) ...
Setting up grub2-common (1.99-27+deb7u1) ...
Setting up grub-pc-bin (1.99-27+deb7u1) ...
Setting up grub-pc (1.99-27+deb7u1) ...
Installation finished. No error reported.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.9.4-zgsrv2008064
Found initrd image: /boot/initrd.img-3.9.4-zgsrv2008064
done
Setting up parted (2.3-12) ...
Setting up dmsetup (2:1.02.74-7) ...
update-initramfs: deferring update (trigger activated)
Setting up lvm2 (2.02.95-7) ...
[ ok ] Setting up LVM Volume Groups...done.
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.9.4-zgsrv2008064
Found 12 processes using old versions of upgraded files
( )
$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libasprintf0c2:i386 libcurl3-gnutls:i386 libfreetype6:i386 libfuse2:i386
  libreadline5:i386
Use  apt-get autoremove  to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
$ dpkg --print-architecture
amd64
$ dpkg --print-foreign-architectures
i386
$ dpkg --list   grep i386   wc -l
72
$ dpkg --list   grep amd64   wc -l
175
$ 
We have completed the actual crossgrade. All non-multiarch packages are now amd64. The resolver is now fine as well. At this current state, all i386 packages that are still installed should be unneeded libs. We can safely nuke them. Careful minds may inspect the list of packages to be removed for important or non-lib items. After completion of the command, a second call of the same command verifies that we were successful.
$ sudo apt-get remove $(dpkg --list   awk  if( $1 ==  ii  && $4 ==  i386  )   print $2
  )
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  e2fslibs:i386 gcc-4.7-base:i386 libacl1:i386 libapt-inst1.5:i386
  libapt-pkg4.12:i386 libasprintf0c2:i386 libattr1:i386 libblkid1:i386
  libbsd0:i386 libbz2-1.0:i386 libc6:i386 libcap2:i386 libcomerr2:i386
  libcurl3:i386 libcurl3-gnutls:i386 libdb5.1:i386 libedit2:i386
  libexpat1:i386 libfreetype6:i386 libfuse2:i386 libgcc1:i386
  libgcrypt11:i386 libgdbm3:i386 libgnutls26:i386 libgpg-error0:i386
  libgpm2:i386 libgssapi-krb5-2:i386 libidn11:i386 libk5crypto3:i386
  libkeyutils1:i386 libkmod2:i386 libkrb5-3:i386 libkrb5support0:i386
  libldap-2.4-2:i386 liblockfile1:i386 liblzma5:i386 libmagic1:i386
  libncurses5:i386 libncursesw5:i386 libp11-kit0:i386 libpam-modules:i386
  libpam0g:i386 libpci3:i386 libpcre3:i386 libpng12-0:i386 libpopt0:i386
  libprocps0:i386 libreadline5:i386 libreadline6:i386 librtmp0:i386
  libsasl2-2:i386 libselinux1:i386 libsemanage1:i386 libsepol1:i386
  libsigc++-2.0-0c2a:i386 libslang2:i386 libsqlite3-0:i386 libss2:i386
  libssh2-1:i386 libssl1.0.0:i386 libstdc++6:i386 libtasn1-3:i386
  libtinfo5:i386 libudev0:i386 libusb-0.1-4:i386 libustr-1.0-1:i386
  libuuid1:i386 libwrap0:i386 linux-image-3.9.4-zgsrv2008064:i386 zlib1g:i386
0 upgraded, 0 newly installed, 70 to remove and 0 not upgraded.
After this operation, 67,5 MB disk space will be freed.
Do you want to continue [Y/n]? y
(Reading database ... 17111 files and directories currently installed.)
Removing e2fslibs:i386 ...
( )
$ sudo apt-get remove $(dpkg --list   awk  if( $1 ==  ii  && $4 ==  i386  )   print $2
  )
Reading package lists... Done
Building dependency tree       
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
$ dpkg --print-architecture
amd64
$ dpkg --print-foreign-architectures
i386
$ dpkg --list   grep i386   wc -l
70
$ dpkg --list   grep amd64   wc -l
175
$ 
To finally make the packages really vanish from the system, we purge all packages that dpkg --list still reports in state rc , which means removed, configuration still present . Thankfully, dpkg seems to handle the case where a dpkg-conffile belongs to a transitioned package gracefully.
$ sudo dpkg --purge $(dpkg --list   awk  if ($1 ==  rc )   print $2  )
(Reading database ... 15868 files and directories currently installed.)
Removing e2fslibs:i386 ...
Purging configuration files for e2fslibs:i386 ...
( )
$ dpkg --print-architecture
amd64
$ dpkg --print-foreign-architectures
i386
$ dpkg --list   grep i386   wc -l
0
$ dpkg --list   grep amd64   wc -l
175
$ 
We now verify that all packages we have are either arch all or arch amd64.
$ dpkg --list   grep -v  \(amd64 all\) 
Desired=Unknown/Install/Remove/Purge/Hold
  Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
 / Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
 / Name                            Version                   Architecture Description
+++-===============================-=========================-============-========================================================================
As I found myself suddenly without e2fsprogs during one of my experiments, I check for e2fsprogs being present. If this package is missing, you ll only find out after rebooting.
$ dpkg --list e2fsprogs
Desired=Unknown/Install/Remove/Purge/Hold
  Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
 / Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
 / Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  e2fsprogs      1.42.5-1.1   amd64        ext2/ext3/ext4 file system utilit
My locally built amd64 kernel with 32 bit support was an arch i386 package as well, so it got zapped in the migration process. Replace it with a real amd64 kernel. This step will vary if you use Debian kernels. Do not forget the firmwares.
$ sudo apt-get -t wheezy-zg-experimental install kernel-image-zgserver linux-firmware-image
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  cpio initramfs-tools kernel-image-zgsrv20080 klibc-utils libklibc
  linux-image-3.9.4-zgsrv20080
Suggested packages:
  libarchive1
The following NEW packages will be installed:
  cpio initramfs-tools kernel-image-zgserver kernel-image-zgsrv20080
  klibc-utils libklibc linux-firmware-image linux-image-3.9.4-zgsrv20080
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 11,2 MB of archives.
After this operation, 30,5 MB of additional disk space will be used.
Do you want to continue [Y/n]? 
Get:1 http://zg20110.debian.zugschlus.de/ wheezy-zg-experimental/main linux-image-3.9.4-zgsrv20080 amd64
3.9.4.20130605.0 [10,4 MB]
( )
I now make sure that the initrd I have built contains all file system modules and is of the right architecture:
$ cd /tmp
$ < /boot/initrd.img-3.9.4-zgsrv20080 gunzip   cpio -i
32053 blocks
$ find . -name  *ext*.ko 
./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext3/ext3.ko
./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext2/ext2.ko
./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext4/ext4.ko
$ file ./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext4/ext4.ko
./lib/modules/3.9.4-zgsrv20080/kernel/fs/ext4/ext4.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV),
BuildID[sha1]=0x259a642448805c9597932ee25c62c8e6a00a32ad, not stripped
$ cd
As the last step, remove i386 as an architecture and reboot into the newly migrated system. We verify that the system considers itself amd64 with no foreign architectures.
$ sudo dpkg --remove-architecture i386
$ dpkg --print-architecture
amd64
$ 
$ dpkg --print-foreign-architectures
$ dpkg --list   grep i386   wc -l
0
$ dpkg --list   grep amd64   wc -l
181
$ sudo shutdown -r now
Broadcast message from root@wheezyarch (pts/0) (Wed Jun 12 16:09:24 2013):
The system is going down for reboot NOW!
$ Connection to wheezyarch.zugschlus.de closed by remote host.
Connection to wheezyarch.zugschlus.de closed.
$ ssh wheezyarch.zugschlus.de
Linux wheezyarch 3.9.4-zgsrv20080 #2 SMP PREEMPT Wed Jun 5 13:12:04 UTC 2013 x86_64
$ dpkg --print-architecture
amd64
$
We now check whether we have lost any packages in this process:
$ sudo dpkg --get-selections   grep install   awk  print $1    sed  s/:.*//;s/$/:amd64
install/;  > get-selections.post
$ wc -l get-selections.*
  234 get-selections.post
  226 get-selections.pre
$ diff --unified=0 get-selections.pre get-selections.post 
--- get-selections.pre  2013-06-12 19:47:40.278176000 +0200
+++ get-selections.post 2013-06-12 20:28:51.389874000 +0200
@@ -22,0 +23 @@
+cpio:amd64 install
@@ -58,0 +60 @@
+initramfs-tools:amd64 install
@@ -68,0 +71,3 @@
+kernel-image-zgserver:amd64 install
+kernel-image-zgsrv20080:amd64 install
+klibc-utils:amd64 install
@@ -108,0 +114 @@
+libklibc:amd64 install
@@ -163,0 +170,2 @@
+linux-firmware-image:amd64 install
+linux-image-3.9.4-zgsrv20080:amd64 install
So we have not actually lost things, but we have gained the pure amd64 kernel and the packages that are needed to build a proper initrd. These were missing from the initial install of the i386 test VM. The downside of the process is that we have lost most of the automatically installed markers in aptitude which make sure to remove a library once nothing depends on it any more. If you make a point in having those markers correctly set, as I do, this is tedious work to set it on every package in the aptitude curses interface, making aptitude suggest removing the entire system, and then again remove it from the packages you actually intend to keep. This should not result in anything being removed, otherwise you have goofed previously. All Done! I am now off perfecting my upgrade process so that the i386 lenny and squeeze machines can first be moved to wheezy and then to amd64.

1 March 2013

Daniel Pocock: Monitoring JBoss, Tomcat and other application servers with JMXetric

JMXetric 1.0 was finally released in February, let's have a quick look at how conveniently it can be integrated with an application server. A more thorough analysis of gmetric4j and JMXetric can be found in the sections I contributed to the new O'Reilly book Monitoring with Ganglia General comments about JVM instrumentation and the JVM boot classpath Monitoring agents (there are a number of them out there) are loaded at an early stage by the JVM and it needs to have access to any of their dependencies through the boot classpath. JMXetric explicitly declares it's dependencies in MANIFEST.MF:
Boot-Class-Path: oncrpc-1.0.7.jar gmetric4j-1.0.2.jar
As no absolute path is specified, the JVM will look for those JARs in the same directory where the jmxetric.jar is located. Furthermore, the JVM will expect the exact filenames to match. If other filenames are used (e.g. for a newer version of gmetric), then the JVM command line needs to specify the filenames and locations, for example:
-Xbootclasspath/a:/opt/ganglia/jmxetric/gmetric4j-1.0.3.jar
Starting JBoss 7 with JMXetric To start with, unpack a vanilla JBoss distribution:
$ mkdir ~/opt
$ cd ~/opt
$ tar xzf ~/Downloads/jboss-as-7.1.0.Final.tar.gz
Now get the JMXetric code and put it, together with dependencies, in the same directory:
$ mkdir -p ~/opt/ganglia/jmxetric
$ cd ~/opt/ganglia/jmxetric
$ wget http://search.maven.org/remotecontent?filepath=info/ganglia/jmxetric/jmxetric/1.0.2/jmxetric-1.0.2.jar
$ wget http://search.maven.org/remotecontent?filepath=info/ganglia/gmetric4j/gmetric4j/1.0.2/gmetric4j-1.0.2.jar
$ wget http://search.maven.org/remotecontent?filepath=org/acplt/oncrpc/1.0.7/oncrpc-1.0.7.jar
Set up a config file for JMXetric:
$ mkdir -p ~/opt/ganglia/jmxetric/etc
$ cd ~/opt/ganglia/jmxetric/etc
$ wget https://raw.github.com/ganglia/jmxetric/master/etc/jmxetric.xml
$ cp jmxetric.xml jmxetric-jboss.xml
$ vi jmxetric-jboss.xml
and insert the following code:
                <!-- report web server request count -->
		<mbean name="jboss.as:subsystem=web,connector=http" pname="http">
			<attribute name="requestCount" type="int32"/>
		</mbean>
Finally, tell JBoss about the agent:
$ cd ~/opt/jboss-as-7.1.0.Final
$ vi bin/standalone.conf
and apply the following changes to standalone.conf:
--- bin/standalone.conf
+++ ../jboss-as-7.1.0.Final.for-jmxetric/bin/standalone.conf
@@ -4,6 +4,7 @@
 ##                                                                          ##
 ##############################################################################
 
+JBOSS_MODULES_SYSTEM_PKGS="org.jboss.logmanager"
 #
 # This file is optional; it may be removed if not needed.
 #
@@ -48,6 +49,8 @@
 #
 if [ "x$JAVA_OPTS" = "x" ]; then
    JAVA_OPTS="-Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"
+   JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Xbootclasspath/p:$JBOSS_HOME/modules/org/jboss/logmanager/main/jboss-logmanager-1.2.2.GA.jar -Xbootclasspath/p:$JBOSS_HOME/modules/org/jboss/logmanager/log4j/main/jboss-logmanager-log4j-1.0.0.GA.jar -Xbootclasspath/p:$JBOSS_HOME/modules/org/apache/log4j/main/log4j-1.2.16.jar"
+   JAVA_OPTS="$JAVA_OPTS -javaagent:$HOME/opt/ganglia/jmxetric/jmxetric-1.0.2.jar=host=jboss-host.example.org,port=8649,wireformat31x=true,config=$HOME/opt/ganglia/jmxetric/etc/jmxetric-jboss.xml"
    JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"
    JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone.xml"
 else
Note that we are adding some of the core JBoss logging code to the boot classpath - that is because JMXetric depends on the logger, and like many monitoring agents, it is impacted by JBoss bug AS-3681. Adding these extra classpath settings is the workaround. Now go ahead and start JBoss, you should see a log message informing you that JMXetric is operational:
$ cd ~/opt/jboss-as-7.1.0.Final
$ chmod a+x bin/*.sh
$ bin/standalone.sh
=========================================================================
  JBoss Bootstrap Environment
  JBOSS_HOME: ...
.
.
.
=========================================================================
JMXetricAgent instrumented JVM, see https://github.com/ganglia/jmxetric
12:59:34,395 INFO  [info.ganglia.gmetric4j.GMonitor] Setting up 1 samplers
12:59:34,536 INFO  [org.jboss.modules] JBoss Modules version 1.1.1.GA
12:59:34,617 INFO  [org.jboss.msc] JBoss MSC version 1.0.2.GA
12:59:34,653 INFO  [org.jboss.as] JBAS015899: JBoss AS 7.1.0.Final "Thunder" starting
.
.
.
I hope this example is useful for you. Please feel free to share recipes you develop for collecting useful JVM metrics, and also any custom charts/reports through collaboration with the Ganglia community mailing list, the Ganglia wiki or your own blog.

26 November 2012

Steve Langasek: Upstart in Debian

Good news, everyone! So as of last Sunday, this works on all Linux archs in Debian unstable and gives you a modern version of upstart:
echo 'Yes, do as I say!'   apt-get -o DPkg::options=--force-remove-essential -y --force-yes install upstart
Thanks to the ifupdown, sysvinit, and udev maintainers for their cooperation in getting upstart support in place; to the Debian release team for accomodating the late changes needed for upstart to be supported in wheezy; and to Scott for his past maintenance of upstart in Debian. Benchmarking One of the consequences is that it's now possible to do meaningful head-to-head comparisons of boot speed between sysvinit (with startpar), upstart, and systemd. At one time or another people have tested systemd vs. sysvinit when using bash as /bin/sh, and upstart vs. sysvinit, and systemd vs. sysvinit+startpar, and there are plenty of bootcharts floating around showing results of one init system or another on one distro or another, but I'm not aware of anyone having done a real, fair comparison of the three solutions, changing nothing but the init system. I've done some initial comparisons in a barebones sid VM, and the results are definitely interesting. Sysvinit with startpar (the default in Debian) can boot a stock sid install, with no added services, in somewhere between 3.37 and 3.42 seconds (three runs). That's not a whole lot, but on the other hand this is a system with a single filesystem and no interesting services yet. Is this really as fast as we can boot? No, even this minimal system can boot faster. Testing with upstart shows that upstart can do the same job in between 3.03 and 3.19 seconds (n=3, mean=3.09). This confirms what we'd already seen in Ubuntu, that it makes a difference to boot speed to have filesystem mounting handled by an integrated process that understands the whole system instead of as a group of serialized shell scripts. What about systemd? The same test gives a boot time between 2.32 and 2.85 seconds (n=4, mean=2.48). Interesting; what would make systemd faster than upstart in sid? Well, a quick look at the system shows one possible contributing factor: the rsyslog package in Debian has a systemd unit file, but not an upstart job file. Dropping in the /etc/init/rsyslog.conf from Ubuntu has a noticeable impact, and brings the upstart boot time down nearer to that of systemd (2.78-3.03s, n=5, mean=2.92). Besides telling me that it's time to start spamming Debian maintainers with wishlist bugs asking them to include upstart jobs in their packages, this suggests that the remaining difference in boot time may be due to the outstanding init scripts in rcS.d that are made built-in no-ops by systemd but not (yet) by upstart in Debian (e.g., hwclock, hostname, udev-mtab). (In Ubuntu, /etc/rcS.d has long since been emptied out in favor of upstart jobs in the common case, since the time it takes to get to runlevel=2 is definitely a major issue for boot speed and boot parallelization.) It also gives the lie to the claim that's been made in various places that spawning shells is a major bottleneck for upstart vs. systemd. More study is certainly needed to confirm this, but at least this naive first test suggests that in spite of the purported benefits of hard-coding boot-time policies in C code, upstart with its default degree of runtime configurability is at least in the ballpark of systemd. Indeed, when OpenSuSE switched from upstart to systemd, it seems that something else in the stack managed to nullify any benefit from improving the boot-time performance of apparmor. Contrary to what some would have you believe, systemd is not some kind of silver bullet for boot speed. Upstart, with its boot-time flexibility and its long history of real-world testing in Ubuntu, is a formidable competitor to systemd in the boot speed department - and a solid solution to the many longstanding boot-time ordering bugs in Debian, which still affect users of sysvinit. I've published the bootcharts for the above tests here. Between the fact that Debian's bootchart package logs by default to /var/log/bootchart.tgz (thus overwriting on every boot) and the fact that these tests are in a VM, I haven't bothered to include the raw data, just the bootcharts themselves. The interested reader can probably generate more interesting boot charts of their own anyway - in particular, it will be interesting to see how the different init systems perform with more complicated filesystem layouts, or when booting a less trivial set of services. Other musings The boot charts have been created with the bootchart package rather than with bootchart2. For one thing, it turns out that bootchart2 includes systemd units, not init scripts; so when replacing bootchart with bootchart2, the non-matching init script is left behind and systemd in particular gets terribly confused. This is now reported as Bug #694403. In an amusing twist, while I was experimenting with bootchart2, I also noticed that having systemd installed would slow down booting with other init systems, because systemd installs udev rules which take a noticeable amount of time to run a helper command at boot even though the helper should be a no-op. So if you're doing boot speed testing of other init systems, be sure you don't have systemd on the system at the time!

5 September 2011

Steve McIntyre: Armhf buildds and porter box hosted at ARM

I'm in the middle of setting up new build machines for the armhf port (see the wiki for more details). We'll shortly have six machines set up in the machine room here at ARM in Cambridge:
  • harris as a porter box
  • hartmann, hasse, hebden and henze running buildd software
  • hoiby as a hot spare
hartmann All of these machines are Freescale i.MX53 Quickstart (aka "loco") development boards. They include a 1GHz i.MX53 CPU (based around the ARM Cortex A8, one of the ARMv7-A family). They have 1GB of RAM and native SATA. They're lovely little machines, measuring just 3 inches square. To mount them usefully in a machine room, I've mounted each board with a 320GB notebook hard drive and the necessary cabling onto a small perspex card as you can see here. Then we can fit 6 such machines and a normal PC-style ATX PSU into a 3U mini-rack. Well, it almost fits - the power supply pokes out a little so we'll need 4U of space when we come to mount it. armhf mini rack The Quickstart boards have been sponsored by Linaro, and ditto my time setting up these machines. Thanks! As is common with new development boards, these machines are not quite fully supported in Debian yet. The kernels we're using are locally-built, using the sources supplied by Freescale. For now, that means a heavily-patched "2.6.35" kernel but we're expecting to be able to switch to mainline very soon. The .config I'm using is kernel.config, and I've built it natively on harris using fakeroot make -j2 deb-pkg DEBEMAIL=93sam@debian.org DEBFULLNAME="Steve McIntyre" KDEB_PKGVERSION=1buildd1 Similarly to the setup for the armel machines, for now I've tweaked things when installing the kernel:
  • depmod:
    • Need to make sure that depmod is run so the new kernel can find and load modules at boot. Added trivial script in /etc/kernel/postinst.d/depmod to do this.
  • initramfs-tools:
    • Needed to copy the file /etc/kernel/postinst.d/initramfs-tools into place from my amd64 machine; I'm guessing this would be there automatically on a new-enough version of initramfs-tools on the armel machines, but we're still using Lenny as a base system for now even if I'm using a Squeeze-based kernel.
  • flash-kernel:
    • Add support for these boards
    • #550584: kernel postinst hook script (/etc/kernel/postinst.d/zz-flash-kernel) to create uImage and uInitrd files from the kernel zImage and the initramfs.
Finally, I've tweaked the uboot config on the machines to use the uImage and uInitrd files that are generated by flash-kernel:
MX53-LOCO U-Boot > setenv loadaddr 0x70800000
MX53-LOCO U-Boot > setenv initrdaddr 0x71000000
MX53-LOCO U-Boot > setenv bootargs_sata set bootargs \$\ bootargs\  root=/dev/sda2 rw rootwait
MX53-LOCO U-Boot > setenv load_sata_kernel ext2load sata 0:1 \$\ loadaddr\  /uImage
MX53-LOCO U-Boot > setenv load_sata_initrd ext2load sata 0:1 \$\ initrdaddr\  /uInitrd
MX53-LOCO U-Boot > setenv load_sata run load_sata_kernel load_sata_initrd
MX53-LOCO U-Boot > setenv bootcmd_sata sata init\; run bootargs_base bootargs_sata load_sata\; bootm \$\ loadaddr\  \$\ initrdaddr\ 
MX53-LOCO U-Boot > setenv bootcmd run bootcmd_sata
And I've added extra config into uboot to use the pre-installed Ubuntu system on the micro SD card as a fall-back:
MX53-LOCO U-Boot > setenv bootcmd_rescue sata init\; run bootargs_base bootargs_sata\; mmc read 0 \$\ loadaddr\  0x800 0x1800\; bootm

1 September 2011

Matthew Garrett: The Android/GPL situation

There was another upsurge in discussion of Android GPL issues last month, triggered by couple of posts by Edward Naughton, followed by another by Florian Mueller. The central thrust is that section 4 of GPLv2 terminates your license on violation, and you need the copyright holders to grant you a new one. If they don't then you don't get to distribute any more copies of the code, even if you've now come into compliance. TLDR; most Android vendors are no longer permitted to distribute Linux.

I'll get to that shortly. There's a few other issues that could do with some clarification. The first is Naughton's insinuation that Google are violating the GPL due to Honeycomb being closed or their "license washing" of some headers. There's no evidence whatsoever that Google have failed to fulfil their GPL obligations in terms of providing source to anyone who received GPL-covered binaries from them. If anyone has some, please do get in touch. Some vendors do appear to be unwilling to hand over code for GPLed bits of Honeycomb. That's an issue with the vendors, not Google.

His second point is more interesting, but the summary is "Google took some GPLed header files and relicensed them under Apache 2.0, and they've taken some other people's GPLv2 code and put it under Apache 2.0 as well". As far as the headers go, there's probably not much to see here. The intent was to produce a set of headers for the C library by taking the kernel headers and removing the kernel-only components. The majority of what's left is just structure definitions and function prototypes, and is almost certainly not copyrightable. And remember that these are the headers that are distributed with the kernel and intended for consumption by userspace. If any of the remaining macros or inline functions are genuinely covered by the GPLv2, any userspace application including them would end up a derived work. This is clearly not the intention of the authors of the code. The risk to Google here is indistinguishable from zero.

How about the repurposing of other code? Naughton's most explicit description is:

For example, Android uses bootcharting logic, which uses the 'bootchartd' script provided by www.bootchart.org, but a C re-implementation that is directly compiled into our init program. The license that appears at www.bootchart.org is the GPLv2, not the Apache 2.0 license that Google claims for its implementation.

, but there's no indication that Google's reimplementation is a derived work of the GPLv2 original.

In summary: No sign that Google's violating the GPL.

Florian's post appears to be pretty much factually correct, other than this bit discussing the SFLC/Best Buy case:

I personally believe that intellectual property rights should usually be enforced against infringing publishers/manufacturers rather than mere resellers, but that's a separate issue.

The case in question was filed against Best Buy because Best Buy were manufacturing infringing devices. It was a set of own-brand Blu Ray players that incorporated Busybox. Best Buy were not a mere reseller.

Anyway. Back to the original point. Nobody appears to disagree that section 4 of the GPLv2 means that violating the license results in total termination of the license. The disagreement is over what happens next. Armijn Hemel, who has done various work on helping companies get back into compliance, believes that simply downloading a new copy of the code will result in a new license being granted, and that he's received legal advice that supports that. Bradley Kuhn disagrees. And the FSF seem to be on his side.

The relevant language in v2 is:

You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License.

The relevant language in v3 is:

You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License

which is awfully similar. However, v3 follows that up with:

However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.

In other words, with v3 you get your license back providing you're in compliance. This doesn't mesh too well with the assumption that you can get a new license by downloading a new copy of the software. It seems pretty clear that the intent of GPLv2 was that the license termination was final and required explicit reinstatement.

So whose interpretation is correct? At this point we really don't know - the only people who've tried to use this aspect of the GPL are the SFLC, and as part of their settlements they've always reinstated permission to distribute Busybox. There's no clear legal precedent. Which makes things a little awkward.

It's not possible to absolutely say that many Android distributors no longer have the right to distribute Linux. But nor is it possible to absolutely say that they haven't lost that right. Any sufficiently motivated kernel copyright holder probably could engage in a pretty effective shakedown racket against Android vendors. Whether they will do remains to be seen, but honestly if I were an Android vendor I'd be worried. There's plenty of people out there who hold copyright over significant parts of the kernel. Would you really bet on all of them being individuals of extreme virtue?

comment count unavailable comments

24 July 2011

Martin Michlmayr: Upgrade to mainline U-Boot from Debian archive

When Marvell originally released the first plug computer, they created their own version of u-boot with support for their new devices. Unfortunately, this version of u-boot is fairly out of date nowadays compared to mainline u-boot and has several problems. Support for plug computers (such as SheevaPlug and GuruPlug) have been integrated into the mainline u-boot (also known as DENX u-boot) in the meantime and Clint Adams has packaged it for Debian. I finally found the time to test Clint's u-boot binary on my devices and have updated the SheevaPlug installation guide accordingly. If you're have installed Debian to a SheevaPlug according to my instructions, I suggest you upgrade. If you boot from a MMC/SD card, you should be aware that the mmcinit command has been renamed to mmc init in order to be consistent with the naming of other commands. You'll therefore have to update your bootcmd_mmc variable in u-boot like this:
setenv bootcmd_mmc 'mmc init; ext2load mmc 0:1 0x00800000 /uImage; ext2load mmc 0:1 0x01100000 /uInitrd'
saveenv

12 June 2011

Clint Adams: Debian on the DreamPlug, Part II

If you have debootstrapped the internal microSD card, the vendor-supplied U-Boot config should serve you adequately. To boot off of a USB stick, however, interrupt the boot sequence (if you don't have a JTAG/serial board at this point, go buy one immediately), and enter the following commands:
setenv x_bootcmd_kernel fatload usb 1:1 0x6400000 uImage
setenv x_bootargs_root root=/dev/sdc2 rootdelay=10
saveenv
reset
If you just want to testboot once, and don't want to make it permanent, omit the saveenv. If you do want to make it permanent, consider removing the superfluous pings from the bootcmd, but be careful to escape semicolons and dollar signs instead of zealously copying and pasting. If you haven't bothered putting a FAT partition with uImage on the USB stick, omit the setenv x_bootcmd_kernel line. Once you've booted Debian, you may wish to natively-debootstrap or copy your new partition to /dev/sda2 (which is what Linux will call the ext3 partition of your microSD card). To then return to fully booting off of microSD so you can repurpose the USB stick,
setenv x_bootcmd_kernel fatload usb 0:1 0x6400000 uImage
setenv x_bootargs_root root=/dev/sda2 rootdelay=10
saveenv
reset
Enjoy.

10 February 2011

Jurij Smakov: My own little Squeeze release

Squeeze installer for sparc is affected by a number of problems, discovered late in the release cycle. In particular:
  • #608516: niu network driver is not included in installer udebs.
  • #610906: pata_cmd64x driver is not included in installer initrd, CD-ROM and hard drives are not detected on systems which require it as a result.
  • #602853: kernel does not boot on machines which need atyfb graphics driver, last message seen on the console is "console [tty0] enabled, bootconsole disabled".
These are documented in Squeeze installation notes, however it would be nice to push available fixes for these bugs into the first point release. In order to test the fixes I've created a couple of unofficial Squeeze installer images for sparc, which can be found here. See the README file for details. If you have a sparc system where install is affected by one of these bugs, it would be great if you could try it out and send a report to debian-sparc or directly to me. There is also #611314 (sym53c8xx module is not loaded automatically), but we don't know what causes it yet. For now just use a simple workaround described in the bug.

Next.