Search Results: "Norbert Preining"

22 November 2017

Norbert Preining: Kobo firmware 4.6.10075 mega update (KSM, nickel patch, ssh, fonts)

A new firmware for the Kobo ebook reader came out and I adjusted the mega update pack to use it. According to the comments in the firmware thread it is working faster than previous releases. The most incredible change though is the update from wpa_supplicant 0.7.1 (around 2010) to 2.7-devel (current). Wow. Kobo Logo For details and warning please consult the previous post. Download Mark6 Kobo GloHD firmware: Kobo 4.6.9995 for GloHD Mega update: Kobo-4.6.10075-combined/Mark6/KoboRoot.tgz Mark5 Aura firmware: Kobo 4.6.9995 for Aura Mega update: Kobo-4.6.10075-combined/Mark5/KoboRoot.tgz Mark4 Kobo Glo, Aura HD firmware: Kobo 4.6.9995 for Glo and AuraHD Mega update: Kobo-4.6.10075-combined/Mark4/KoboRoot.tgz Enjoy.

17 November 2017

Norbert Preining: ScalaFX: Problems with Tables abound

Doing a lot with all kinds of tables in ScalaFX, I stumbled upon a bug in ScalaFX that, with the help of the bug report, I was able to circumvent. It is a subtle bug where types are mixed between scalafx.SOMETHING and the corresponding javafx.SOMETHING.
In one of the answers it is stated that:
The issue is with implicit conversion from TableColumn not being located by Scala. I am not clear why this is happening (maybe a Scala bug).
But the provided work-around at least made it work. Until today I stumbled onto a (probably) just another instance of this bug, but where the same work-around does not help. I am using TreeTableViews and try to replace the children of the root by filtering out one element. The code I use is of course very different, but here is a reduced and fully contained example, based on the original bug report and adapted to use a TreeTableView:
import scalafx.Includes._
import scalafx.scene.control.TreeTableColumn._
import scalafx.scene.control.TreeItem._
import scalafx.application.JFXApp.PrimaryStage
import scalafx.application.JFXApp
import scalafx.scene.Scene
import scalafx.scene.layout._
import scalafx.scene.control._
import scalafx.scene.control.TreeTableView
import scalafx.scene.control.Button
import scalafx.scene.paint.Color
import scalafx.beans.property. ObjectProperty, StringProperty 
import scalafx.collections.ObservableBuffer
// TableTester.scala
object TableTester extends JFXApp  
  val characters = ObservableBuffer[Person](
    new Person("Peggy", "Sue", "123", Color.Violet),
    new Person("Rocky", "Raccoon", "456", Color.GreenYellow),
    new Person("Bungalow ", "Bill", "789", Color.DarkSalmon)
  )
  val table = new TreeTableView[Person](
    new TreeItem[Person](new Person("","","",Color.Red))  
      expanded = true
      children = characters.map(new TreeItem[Person](_))
     )  
    columns ++= List(
      new TreeTableColumn[Person, String]  
        text = "First Name"
        cellValueFactory =  
          _.value.value.value.firstName
         
        prefWidth = 180
       ,
      new TreeTableColumn[Person, String]()  
        text = "Last Name"
        cellValueFactory =  
          _.value.value.value.lastName
         
        prefWidth = 180
       
    )
   
  stage = new PrimaryStage  
    title = "Simple Table View"
    scene = new Scene  
      content = new VBox()  
        children = List(
          new Button("Test it")  
            onAction = p =>  
              val foo: ObservableBuffer[TreeItem[Person]] = table.root.value.children.map(p =>  
                val bar: TreeItem[Person] = p
                p
               )
              table.root.value.children = foo
             
           ,
          table)
       
     
   
 
// Person.scala
class Person(firstName_ : String, lastName_ : String, phone_ : String, favoriteColor_ : Color = Color.Blue)  
  val firstName = new StringProperty(this, "firstName", firstName_)
  val lastName = new StringProperty(this, "lastName", lastName_)
  val phone = new StringProperty(this, "phone", phone_)
  val favoriteColor = new ObjectProperty(this, "favoriteColor", favoriteColor_)
  firstName.onChange((x, _, _) => System.out.println(x.value))
 
With this code what one gets on compilation with the latest Scala and ScalaFX is:
[error]  found   : scalafx.collections.ObservableBuffer[javafx.scene.control.TreeItem[Person]]
[error]  required: scalafx.collections.ObservableBuffer[scalafx.scene.control.TreeItem[Person]]
[error]               val foo: ObservableBuffer[TreeItem[Person]] = table.root.value.children.map(p =>  
[error]                                                                                          ^
[error] one error found
And in this case, adding import statements didn t help, what a pity. Unfortunately this bug is open since 2014 with a helpwanted tag and nothing is going on. I guess I have to try to dive into the source code of ScalaFX

10 November 2017

Norbert Preining: ScalaFX: dynamic update of context menu of table rows

Context menus are useful to exhibit additional functionality. For my TLCockpit program I am listing the packages, updates, and available backups in a TreeTableView. The context for each row should be different depending on the status of the content displayed.
My first try, taken from searches on the web, was to add the context menu via the rowFactory of the TreeTableView:
table.rowFactory =   p =>
  val row = new TreeTableRow[SomeObject]  
  val infoMI = new MenuItem("Info")   onAction = /* use row.item.value */  
  val installMI = new MenuItem("Install")   onAction = /* use row.item.value */  
  val removeMI = new MenuItem("Remove")   onAction = /* use row.item.value */  
  val ctm = new ContextMenu(infoMI, installMI, removeMI)
  row.contextMenu = ctm
  row
 
This worked nicely until I tried to disable/enable some items based on the status of the displayed package:
  ...
  val pkg: SomeObject = row.item.value
  val isInstalled: Boolean = /* determine installation status of pkg */
  val installMI = new MenuItem("Install")   
    disable = isInstalled
    onAction = /* use row.item.value */
   
What I did here is just pull the shown package, get its installation status, and disable the Install context menu entry if it is already installed. All good and fine I thought, but somehow reality was different. First there where NullPointerExceptions (rare occurrence in Scala for me), and then somehow that didn t work out at all. The explanation is quite simple to be found by printing something in the rowFactory function. There are only as many rows made as fit into the current screen size (plus a bit), and their content is dynamically updated when one scrolls. But the enable/disable status of the context menu entries were not properly updated. To fix this one needs to add a callback on the displayed item, which is exposed in row.item. So the correct code is (assuming that a SomeObject has a BooleanProperty installed):
table.rowFactory =   p =>
  val row = new TreeTableRow[SomeObject]  
  val infoMI = new MenuItem("Info")   onAction = /* use row.item.value */  
  val installMI = new MenuItem("Install")   onAction = /* use row.item.value */   
  val removeMI = new MenuItem("Remove")   onAction = /* use row.item.value */  
  val ctm = new ContextMenu(infoMI, installMI, removeMI)
  row.item.onChange   (_,_,newContent) =>
    if (newContent != null)  
      val isInstalled: /* determine installation status from newContent */
      installMI.disable = is_installed
      removeMI.disable = !is_installed
     
   
  row.contextMenu = ctm
  row
 
The final output then gives me:
That s it, the context menus are now correctly adapted to the displayed content. If there is a simpler way, please let me know.

31 October 2017

Norbert Preining: Debian/TeX Live 2017.20171031-1

Halloween is here, time to upload a new set of scary packages of TeX Live. About a month has passed, so there is the usual big stream up updates. There was actually an intermediate release to get out some urgent fixes, but I never reported the news here. So here are the accumulated changes and updates. My favorite this time is wallcalendar, a great class to design all kind of calendars, it looks really well done. I immediately will start putting one together. On the font side there is the new addition coelacanth. To quote from the README: Coelacanth is inspired by the classic Centaur type design of Bruce Rogers, described by some as the most beautiful typeface ever designed. It aims to be a professional quality type family for general book typesetting. And indeed it is beautiful! Other noteworthy addition is the Spark font that allows creating sparklines in the running text with LaTeX. Enjoy. New packages algobox, amscls-doc, beilstein, bib2gls, coelacanth, crossreftools, dejavu-otf, dijkstra, ducksay, dynkin-diagrams, eqnnumwarn, fetchcls, fixjfm, glossaries-finnish, hagenberg-thesis, hecthese, ifxptex, isopt, istgame, ku-template, limecv, mensa-tex, musicography, na-position, notestex, outlining, pdfreview, spark-otf, spark-otf-fonts, theatre, unitn-bimrep, upzhkinsoku, wallcalendar, xltabular. Updated packages acmart, amsmath, animate, arabluatex, arara, babel, babel-french, bangorexam, baskervillef, beebe, biblatex-philosophy, biblatex-source-division, bibletext, bidi, bxjaprnind, bxjscls, bxpapersize, bytefield, classicthesis, cochineal, complexity, cooking-units, curves, datetime2-german, dccpaper, doclicense, docsurvey, eledmac, epstopdf, eqparbox, esami, etoc, fbb, fei, fithesis, fmtcount, fnspe, fonts-tlwg, fontspec, genealogytree, glossaries, glossaries-extra, hecthese, hepthesis, hvfloat, ifplatform, ifptex, inconsolata, jfmutil, jsclasses, ketcindy, knowledge, koma-script, l3build, l3experimental, l3kernel, l3packages, langsci, latex2man, latexbug, lato, leadsheets, libertinust1math, listofitems, luatexja, luatexko, luatodonotes, lwarp, markdown, mcf2graph, media9, newtx, novel, numspell, ocgx2, overpic, philokalia, phonenumbers, platex, poemscol, pst-exa, pst-geometrictools, pst-ovl, pst-plot, pst-pulley, pst-tools, pst-vehicle, pst2pdf, pstool, pstricks, pstricks-add, pxchfon, pxjahyper, quran, randomlist, rec-thy, reledmac, robustindex, scratch, skrapport, spectralsequences, tcolorbox, tetex, tex4ht, texcount, texdoc, tikzducks, tikzsymbols, toptesi, translation-biblatex-de, unicode-math, updmap-map, uplatex, widetable, xcharter, xepersian, xetexko, xetexref, xsim, zhlipsum.

30 October 2017

Norbert Preining: Inconsistent version numbers in subversion

Update 2017-10-31: The developers of Subversion were a bit surprised, but it turned out to be a genuine bug. The bug has been fixed the same day. The TeX Live project uses Subversion as its main repository. Reasons for that there are many, see my blog post on svn and git. But today I realized that subversion is broken, badly broken, because one can easily end up with inconsistent version numbers. To understand why this is a problem, I restart from the already linked blog: Our packages have revision numbers based on the latest change revision of all the contained files. Since subversion has a linear history and central repository, this guarantees that in contrast to version numbers provided by authors the revision numbers are strictly increasing . Well, it turned out that this assumption was wrong. Because it is easy to trick subversion to have wrong revision numbers for files. Consider the following flow: In our case we have files listing other packages, and they are getting moved around at times. In this case one package was moved from one collection to another and back. Now, if user A does svn up between revision A+n and A+n+m, then the final revision in his checkout of file xxx will be A+n+m. But a user B that does NOT do svn up between A+n and A+n+m will end up with a final revision of A for file xxx in his checkout. And boooom, revisions are different, although the files are in sync with the server. I cannot grasp what the developers of subversion thought about here, but I consider it deeply broken by design. One more reason the rewrite all our distribution scripts to work with git instead of subversion.

19 October 2017

Norbert Preining: Analysing Debian packages with Neo4j

I just finished the presentation at the Neo4j Online Meetup on getting the Debian UDD into a Neo4j graph database. Besides the usual technical quibbles it did work out quite well. The code for pulling the data from the UDD, as well as converting and importing it into Neo4j is available on Github Debian-Graph. The slides are also available on Github: preining-debian-packages-neo4j.pdf. There are still some things I want to implement, time permitting, because it would be a great tool for better integration for Debian. In any case, graph databases are lots of fun to play around.

18 October 2017

Norbert Preining: Kobo firmware 4.6.9995 mega update (KSM, nickel patch, ssh, fonts)

It has been ages that I haven t updated the MegaUpdate package for Kobo. Now that a new and seemingly rather bug-free and quick firmware release (4.6.9995) has been released, I finally took the time to update the whole package to the latest releases of all the included items. The update includes all my favorite patches and features: Kobo Start Menu, koreader, coolreader, pbchess, ssh access, custom dictionaries, and some side-loaded fonts. Kobo Logo So what are all these items: Install procedure Download Mark6 Kobo GloHD firmware: Kobo 4.6.9995 for GloHD Mega update: Kobo-4.6.9995-combined/Mark6/KoboRoot.tgz Mark5 Aura firmware: Kobo 4.6.9995 for Aura Mega update: Kobo-4.6.9995-combined/Mark5/KoboRoot.tgz Mark4 Kobo Glo, Aura HD firmware: Kobo 4.6.9995 for Glo and AuraHD Mega update: Kobo-4.6.9995-combined/Mark4/KoboRoot.tgz Latest firmware Warning: Sideloading or crossloading the incorrect firmware can break/brick your device. The link below is for Kobo GloHD ONLY. The first step is to update the Kobo to the latest firmware. This can easily be done by just getting the latest firmware from the links above and unpacking the zip file into the .kobo directory on your device. Eject and enjoy the updating procedure. Mega update Get the combined KoboRoot.tgz for your device from the links above and put it into the .kobo directory, then eject and enjoy the updating procedure again. After this the device should reboot and you will be kicked into KSM, from where after some time of waiting Nickel will be started. If you consider the fonts too small, select Configure, then the General, and add item, then select kobomenuFontsize=55 and save. Remarks to some of the items included The full list of included things is above, here are only some notes about what specific I have done. WARNINGS If this is the first time you install this patch, you to fix the password for root and disable telnet. This is an important step, here are the steps you have to take (taken from this old post):
  1. Turn on Wifi on the Kobo and find IP address
    Go to Settings Connect and after this is done, go to Settings Device Information where you will see something like
    IP Address: 192.168.1.NN

    (numbers change!)
  2. telnet into your device
    telnet 192.168.1.NN
    it will ask you the user name, enter root (without the quotes) and no password
  3. (ON THE GLO) change home directory of root
    edit /etc/passwd with vi and change the entry for root by changing the 6th field from: / to /root (without the quotes). After this procedure the line should look like
    root::0:0:root:/root:/bin/sh
    don t forget to save the file
  4. (ON THE GLO) create ssh keys for dropbear
    [root@(none) ~]# mkdir /etc/dropbear
    [root@(none) ~]# cd /etc/dropbear
    [root@(none) ~]# dropbearkey -t dss -f dropbear_dss_host_key
    [root@(none) ~]# dropbearkey -t rsa -f dropbear_rsa_host_key
  5. (ON YOUR PERSONAL COMPUTER) check that you can log in with ssh
    ssh root@192.168.1.NN
    You should get dropped into your device again
  6. (ON THE GLO) log out of the telnet session (the first one you did)
    [root@(none) ~]# exit
  7. (ON THE GLO) in your ssh session, change the password of root
    [root@(none) ~]# passwd
    you will have to enter the new password two times. Remember it well, you will not be easily able to recover it without opening your device.
  8. (ON THE GLO) disable telnet login
    edit the file /etc/inetd.conf.local on the GLO (using vi) and remove the telnet line (the line starting with 23).
  9. restart your device
The combined KoboRoot.tgz is provided without warranty. If you need to reset your device, don t blame me!

17 October 2017

Norbert Preining: Japanese TeX User Meeting 2017

Last saturday the Japanese TeX User Meeting took place in Fujisawa, Kanagawa. For those who have been at the TUG 2013 in Tokyo you will remember that the Japanese TeX community is quite big and vibrant. On Saturday about 50 users and developers gathered for a set of talks on a variety of topics. The first talk was by Keiichiro Shikano ( ) on using Markup text to generate (La)TeX and HTML. He presented a variety of markup formats, including his own tool xml2tex. The second talk was my Masamichi Hosoda ( ) on reducing the size of PDF files using PDFmark extraction. As a contributor to many projects including Texinfo and LilyPond, Masamichi Hosoda tells us horror stories about multiple font embedding in the manual of LilyPond, the permanent need for adaption to newer Ghostscript versions, and the very recent development in Ghostscript prohibiting the merge of font definitions in PDF files. Next up was Yusuke Terada ( ) on grading exams using TeX. Working through hundreds and hundreds of exams and do the grading is something many of us are used to and I think nobody really enjoys it. Yusuke Terada has combined various tools, including scans, pdf merging using pdfpages, to generate gradable PDF which were then checked on an iPad. On the way he did hit some limits in dvipdfmx on the number of images, but this was obviously only a small bump on the road. Now if that could be automatized as a nice application, it would be a big hit I guess! The forth talk was by Satoshi Yamashita ( ) on the preparation of slides using KETpic. KETpic is a long running project by Setsuo Takato ( ) for the generation of graphics, in particular using Cinderella. KETpic and KETcindy integrates with lots of algebraic and statistical programs (R, Maxima, SciLab, ) and has a long history of development. Currently there are activities to incorporate it into TeX Live. The fifth talk was by Takuto Asakura ( ) on programming TeX using expl3, the main building block of the LaTeX3 project and already adopted by many TeX developers. Takuto Asakura came to fame on this years TUG/BachoTeX 2017 when he won the W. J. Martin Prize for his presentation Implementing bioinformatics algorithms in TeX. I think we can expect great new developments from Takuto! The last talk was by myself on fmtutil and updmap, two of the main management programs in any TeX installation, presenting the changes introduced over the last year, including the most recent release of TeX Live. Details have been posted on my blog, and a lengthy article in TUGboat 38:2, 2017 is available on this topic, too. After the conference about half of the participants joined a social dinner in a nearby Izakaya, followed by a after-dinner beer tasting at a local craft beer place. Thanks to Tatsuyoshi Hamada for the organization. As usual, the Japanese TeX User Meetings are a great opportunity to discuss new features and make new friends. I am always grateful to be part of this very nice community! I am looking forward to the next year s meeting.

16 October 2017

Norbert Preining: Fixing vim in Debian

I was wondering for quite some time why on my server vim behaves so stupid with respect to the mouse: Jumping around, copy and paste wasn t possible the usual way. All this despite having
  set mouse=
in my /etc/vim/vimrc.local. Finally I found out why, thanks to bug #864074 and fixed it. The whole mess comes from the fact that, when there is no ~/.vimrc, vim loads defaults.vim after vimrc.local and thus overwriting several settings put in there. There is a comment (I didn t see, though) in /etc/vim/vimrc explaining this:
" Vim will load $VIMRUNTIME/defaults.vim if the user does not have a vimrc.
" This happens after /etc/vim/vimrc(.local) are loaded, so it will override
" any settings in these files.
" If you don't want that to happen, uncomment the below line to prevent
" defaults.vim from being loaded.
" let g:skip_defaults_vim = 1
I agree that this is a good way to setup vim on a normal installation of Vim, but the Debian package could do better. The problem is laid out clearly in the bug report: If there is no ~/.vimrc, settings in /etc/vim/vimrc.local are overwritten. This is as counterintuitive as it can be in Debian and I don t know any other package that does it in a similar way. Since the settings in defaults.vim are quite reasonable, I want to have them, but only fix a few of the items I disagree with, like the mouse. At the end what I did is the following in my /etc/vim/vimrc.local:
if filereadable("/usr/share/vim/vim80/defaults.vim")
  source /usr/share/vim/vim80/defaults.vim
endif
" now set the line that the defaults file is not reloaded afterwards!
let g:skip_defaults_vim = 1
" turn of mouse
set mouse=
" other override settings go here
There is probably a better way to get a generic load statement that does not depend on the Vim version, but for now I am fine with that.

15 October 2017

Norbert Preining: TeX Live Manager: JSON output

With the development of TLCockpit continuing, I found the need for and easy exchange format between the TeX Live Manager tlmgr and frontend programs like TLCockpit. Thus, I have implemented JSON output for the tlmgr info command. While the format is not 100% stable I might change some thing I consider it pretty settled. The output of tlmgr info --data json is a JSON array with JSON objects for each package requested (default is to list all).
[ TLPackageObj, TLPackageObj, ... ]
The structure of the JSON object TLPackageObj reflects the internal Perl hash. Guaranteed to be present keys are name (String) and avilable (Boolean). In case the package is available, there are the following further keys sorted by their type: A rather long example showing the output for the package latex, formatted with json_pp and having the list of files and the long description shortened:
[
    
      "installed" : true,
      "doccontainerchecksum" : "5bdfea6b85c431a0af2abc8f8df160b297ad73f6a324ca88df990f01f24611c9ae80d2f6d12c7b3767308fbe3de3fca3d11664b923ea4080fb13fd056a1d0c3d",
      "docfiles" : [
         "texmf-dist/doc/latex/base/README.txt",
         ....
         "texmf-dist/doc/latex/base/webcomp.pdf"
      ],
      "containersize" : 163892,
      "depends" : [
         "luatex",
         "pdftex",
         "latexconfig",
         "latex-fonts"
      ],
      "runsize" : 414,
      "relocated" : false,
      "doccontainersize" : 12812184,
      "srcsize" : 752,
      "revision" : 43813,
      "srcfiles" : [
         "texmf-dist/source/latex/base/alltt.dtx",
         ....
         "texmf-dist/source/latex/base/utf8ienc.dtx"
      ],
      "category" : "Package",
      "cataloguedata" :  
         "version" : "2017/01/01 PL1",
         "topics" : "format",
         "license" : "lppl1.3",
         "date" : "2017-01-25 23:33:57 +0100"
       ,
      "srccontainerchecksum" : "1d145b567cf48d6ee71582a1f329fe5cf002d6259269a71d2e4a69e6e6bd65abeb92461d31d7137f3803503534282bc0c5546e5d2d1aa2604e896e607c53b041",
      "postactions" : [],
      "binsize" :  ,
      "longdesc" : "LaTeX is a widely-used macro package for TeX, [...]",
      "srccontainersize" : 516036,
      "containerchecksum" : "af0ac85f89b7620eb7699c8bca6348f8913352c473af1056b7a90f28567d3f3e21d60be1f44e056107766b1dce8d87d367e7f8a82f777d565a2d4597feb24558",
      "executes" : [],
      "binfiles" :  ,
      "name" : "latex",
      "catalogue" : null,
      "docsize" : 3799,
      "available" : true,
      "runfiles" : [
         "texmf-dist/makeindex/latex/gglo.ist",
         ...
         "texmf-dist/tex/latex/base/x2enc.dfu"
      ],
      "shortdesc" : "A TeX macro package that defines LaTeX"
    
]
What is currently not available via tlmgr info and thus also not via the JSON output is access to virtual TeX Live databases with several member databases (multiple repositories). I am thinking about how to incorporate this information. These changes are currently available in the tlcritical repository, but will enter proper TeX Live repositories soon. Using this JSON output I will rewrite the current TLCockpit tlmgr interface to display more complete information.

14 October 2017

Norbert Preining: ScalaFX: ListView with CellFactory

I had a bit hard time to get ScalaFX to display a list of items in a scrollable space, and each item can be clicked. I use this in TLCockpit to display the list of documentation files in a TeX Live package, and open it directly from the application. Unfortunately there is not a huge amount of examples using ScalaFX out there in the web, so it took me a bit. My first try was using a VBox with various Labels in there, but this is not scrollable. In other areas I have used TreeTableView, so in this case using ListView should be fine. What I finally came up is the following code:
import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.collections.ObservableBuffer
import scalafx.geometry.Orientation
import scalafx.scene.control. ListCell, ListView 
import scalafx.scene.input.MouseEvent
import scalafx.scene. Cursor, Scene 
import scalafx.scene.paint.Color
import scalafx.Includes._
object ApplicationMain extends JFXApp  
  val SomeStrings: Seq[String] = Seq("Hello", "World", "Enjoy")
  stage = new PrimaryStage  
    title = "ListViewExample"
    scene = new Scene  
      root =  
        new ListView[String]  
          orientation = Orientation.Vertical
          cellFactory =  
            p =>  
              val cell = new ListCell[String]
              cell.textFill = Color.Blue
              cell.cursor = Cursor.Hand
              cell.item.onChange   (_, _, str) => cell.text = str  
              cell.onMouseClicked =   me: MouseEvent => println("Do something with " + cell.text.value)  
              cell
             
           
          items = ObservableBuffer(SomeStrings)
         
       
     
   
 
Some comments to the code, at least as far I understand it: Once managed, it doesn t look so complicated, but took me some time.

13 October 2017

Norbert Preining: Scala: parse JSON into nested case classes

I was playing around with parsing JSON in Scala, and got spray-json as recommendation from my Senpai. My aim was parsing some nested structure like:
  
  "key1" : "value1",
  "key2" :   "subkey1" : "subval1", "subkey2" : "subval2"  
 
into a nested Scala case class:
case class SubClass(subkey1: String, subkey2: String)
case class MainClass(key1: String, key2: SubClass)

To achieve this one needs to define one s own JsonProtocol that describes the parsing:
import spray.json._
object MyJsonProtocol extends DefaultJsonProtocol  
  implicit val subclassFormat = jsonFormat2(SubClass)
  implicit val mainclassFormat = jsonFormat2(MainClass)
 
import MyJsonProtocol._
Using this parsing can be done in the usual way:
val jsonstr = """
      
      "key1" : "value1",
      "key2" :   "subkey1" : "subval1", "subkey2" : "subval2"  
     
    """
val jsonAst = jsonstr.parseJson
val gotit = jsonAst.convertTo[MainClass]
If the input is from a JSON array of the above arrays:
[
    
    "key1" : "value1",
    "key2" :   "subkey1" : "subval1", "subkey2" : "subval2"  
   ,
    
    "key1" : "value10",
    "key2" :   "subkey1" : "subval10", "subkey2" : "subval20"  
   
]
the only change necessary is to call
val gotit = jsonAst.convertTo[List[MainClass]]

04 October 2017

Norbert Preining: Einstein and Freud s letters on Why War? 85th anniversary

85 years ago, on 30 July 1932, Albert Einstein send a letter to Sigmund Freud discussing the question: Why War? Freud answered to this letter in early September 1932. To commemorate the 85 year anniversary, the German typographer Harald Geisler has started a project on Kickstarter to recreate the letters send back then. Over the last weeks the two letters have arrived at my place in Japan:
But not only were the letters reproduces, but typeset in the original digitized handwriting and sent from the original locations. Harald Geisler crafted fonts based on the hand-writing of Einstein and Freud, an layout out the pages of the letters according to the originals. Since the letters were originally written in German, an English translation also typeset in the hand-writing font was added.
In addition to the a bit archaic hand writing of Freud which even many German natives will not be able to read, the German text of his letter has been included in normal print style. Not only that, Harald Geisler even managed to convince the Sigmund Freud Museum to let rest the letters for one night in the very office where the original letter was written, so all the letters send out actually came from Freud s office.
This project was one of the first Kickstarter projects I supported, and I really liked the idea, and would like to thanks Harald Geisler for realizing it. These kind of activities, combining of typography, history, action, dedication, keep our culture and history alive. Thanks.
Harald Geisler also invites us all to continue the dialog on Why War?, which is getting more and more relevant again, with war-mongering becoming respected practice.

26 September 2017

Norbert Preining: Debian/TeX Live 2017.20170926-1

A full month or more has past since the last upload of TeX Live, so it was high time to prepare a new package. Nothing spectacular here I have to say, two small bugs fixed and the usual long list of updates and new packages. From the new packages I found fontloader-luaotfload and interesting project. Loading fonts via lua code in luatex is by now standard, and this package allows for experiments with newer/alternative font loaders. Another very interesting new-comer is pdfreview which lets you set pages of another PDF on a lined background and add notes to it, good for reviewing. Enjoy. New packages abnt, algobox, beilstein, bib2gls, cheatsheet, coelacanth, dijkstra, dynkin-diagrams, endofproofwd, fetchcls, fixjfm, fontloader-luaotfload, forms16be, hithesis, ifxptex, komacv-rg, ku-template, latex-refsheet, limecv, mensa-tex, multilang, na-box, notes-tex, octave, pdfreview, pst-poker, theatre, upzhkinsoku, witharrows. Updated packages 2up, acmart, acro, amsmath, animate, babel, babel-french, babel-hungarian, bangorcsthesis, beamer, beebe, biblatex-gost, biblatex-philosophy, biblatex-source-division, bibletext, bidi, bpchem, bxjaprnind, bxjscls, bytefield, checkcites, chemmacros, chet, chickenize, complexity, curves, cweb, datetime2-german, e-french, epstopdf, eqparbox, esami, etoc, fbb, fithesis, fmtcount, fnspe, fontspec, genealogytree, glossaries, glossaries-extra, hvfloat, ifptex, invoice2, jfmutil, jlreq, jsclasses, koma-script, l3build, l3experimental, l3kernel, l3packages, latexindent, libertinust1math, luatexja, lwarp, markdown, mcf2graph, media9, nddiss, newpx, newtx, novel, numspell, ocgx2, philokalia, phfqit, placeat, platex, poemscol, powerdot, pst-barcode, pst-cie, pst-exa, pst-fit, pst-func, pst-geometrictools, pst-ode, pst-plot, pst-pulley, pst-solarsystem, pst-solides3d, pst-tools, pst-vehicle, pst2pdf, pstricks, pstricks-add, ptex-base, ptex-fonts, pxchfon, quran, randomlist, reledmac, robustindex, scratch, skrapport, spectralsequences, tcolorbox, tetex, tex4ht, texcount, texdef, texinfo, texlive-docindex, texlive-scripts, tikzducks, tikzsymbols, tocloft, translations, updmap-map, uplatex, widetable, xepersian, xetexref, xint, xsim, zhlipsum.

28 August 2017

Norbert Preining: Gaming: The Long Dark Wintermute Episode 1

One month ago the Story Mode for The Long Dark has been released. Finally I managed to finish the first of the two chapters, and it was worth the wait. I have played many ours in the Sandbox mode, but while the first days in story mode are like hand-holding kids, on the fifth day you are kicked into a brutal void with lots, and I mean *lots* of wolves being more than eager to devour you. Targeting new players the four days one just practice basic techniques, fire making, water, food, first aid, collecting materials. Finally one is allowed to leave the crash site of the plane and climbs out into a mountainous area searching for shelter and at the end for Milton, a deserted village. But the moment one reaches a street, wolves are appearing again and again, and the only way is often to run from car to car and hide there, hoping not to freeze to death. Took me several tries to make it to the church and then into town to meet the Grey Mother. The rest of epsiode one is dedicated to various tasks set by the Grey Mother, and some additional (optional) side quests. And while the first encounter with the wolves was pretty grim, in the later parts I had the feeling that they became a bit more easy to take. This might be related to one of the many patches (8 till now) that were shipped out in this month. After having finished all the quests, including the side quests (and found the very useful distress pistol), I made my way out of Milton, probably not to be seen again?! And while all the quests were finished, I had the feeling I could spend a bit more time and explore the surroundings, maybe something is still hiding out there. But shouldering the climbing rope and climbing out of Milton leads to a very beautiful last stretch, a canyon lined with waterfalls leading to a cave and to the next episode. Now I only need more time to play Episode 2.

22 August 2017

Norbert Preining: Debian packages as Graph Database

I have been playing around with Neo4j as graph database, and searching for a big dataset I decided to look at Debian packages (source and binary) from stable, testing, sid, and experimental, and represent all of that in a big graph database. While this is far from ready, the following entities and relations a represented: The graph currently has 220618 nodes and 782323 edges, and my first trial to import this into the database was by generating a long cypher statement, and then throwing that at cypher-shell. Well, that was not the best idea. After 24h I stopped the process and rewrote the generation script to generate csv files. Using neo4j-import the same amount of data was imported in 5secs (!!!). What I would like to get in the future is the whole package history as well, and maybe also include all the bugs into the database if I only would have easily accessible and parseable information about these items (Debian Q&A maybe?). If you have any suggestions, please let me know. More to come, stay tuned.

31 July 2017

Norbert Preining: Calibre in Debian

Some news about Calibre in Debian: I have been added to the list of maintainers, thanks Martin, and the recent release of Calibre 3.4 into Debian/unstable brought some fixes concerning the desktop integration. Now I am working on Calibre 3.5. Calibre 3.5 separates out one module, html5-parser, into a separate package which needs to be included into Debian first. I have prepared and uploaded a version, but NEW processing will keep this package from entering Debian for a while. Other things I am currently doing is going over the list of bugs and try to fix or close as many as possible. Finally the endless Rar support story still continues. I still don t have any response from the maintainer of unrar-nonfree in Debian, so I am contemplating to package my own version of libunrar. As I wrote in the previous post, Calibre now checks whether the Python module unrardll is available, and if, uses it to decode rar-packed ebooks. I have a package for unrardll ready to be uploaded, but it needs the shared library version, and thus I am stuck waiting for unrar-nonfree. Anyway, to help all those wanting to play with the latest Calibre, my archive nowadays contains: Together these packages provide the newest Calibre with Rar support.
deb http://www.preining.info/debian/ calibre main
deb-src http://www.preining.info/debian/ calibre main
The releases are signed with my Debian key 0x6CACA448860CDC13 Enjoy

29 July 2017

Norbert Preining: Gaming: The Long Dark

I normally don t play survival games or walking simulators, but The Long Dark by Hinterland Games, which I purchased back then when it was still in early access on Steam, took me into new realms. You are tossed out into the Canadian wilderness with hardly anything, and your only aim is to survive, find shelter, food, craft tools, hunt for food, explore. And while everything by now is Sandbox mode, on August 1st the first episode of Story mode is released. Best time to get the game! You will be greeted with some icy nights, but also with great vistas, relaxed evenings at a fire place, you will try to survive on moldy food and rotten energy bars, but also feast on the fireside while reading a good book. A real treat this game! Sandbox mode features five different areas to explore. Each one is large enough to spend weeks (in game time) wandering around. The easiest area to start with is Mystery Lake, with plenty of shelter (several huts) and abundance of resources. And just in case you are getting bored, all the areas are connected via tunnels or caves and one can wander of into the neighboring places. My home in Mystery Lake was always the Camp Office, the usual suspect. Nice views, fishing huts nearby to get fresh fish, lots of space. After managing to get from your starting place (which is arbitrary as far as I see) to one of the shelters, one starts collecting food, wood, savaging every accessible place for tools, weapons, burning material. And soon the backpack becomes to heavy, and one needs to store stuff and decide what to take. This is a very well done part of the game. The backpack is not limited by number of items, but you are limited in weight you can carry. That includes clothes (which can get quite heavy) and all the items in your backpack. In addition, the longer the day and the more tired you become, the less weight you can carry. And if the backpack starts getting too heavy you crawl to a very slow movement. There are many influences of the outside world on the player s condition: temperature, the wetness of your clothes, hunger, thirst, exhaustion, but also infections and bruises, all need to be taken care of, otherwise the end is coming faster than one wishes for. I have only two things to complain: First, if one walks outside, or runs outside, the own body temperature does not rise. This is unrealistically and should have been taken into account. The other thing is the difficulty: I have played weeks in game time in the easiest level, without any problem. But the moment I switched to the second level of difficulty (of 5!), I not even manage it for 2(!) days. Wolves, starvation, thirst, any of those kills me within an instant. I don t want to know how the hardest level feels, but it has a certain steep step here. The game takes a very realistic view onto the weather: Every day is different, sunny, foggy, blizzard, windy, often changing very quickly. It is wise to plan one s activities according to the weather, as it is very unforgiving. With beautifully crafted landscapes, loads of areas to explore, your own pride to survive for at least a few weeks, and lots of tools to find and craft and try out, this games, even while it is still in Sandbox mode, is a real feat. My absolute favorite since I have finished the Talos Principle and Portal series, absolutely recommendable!

26 July 2017

Norbert Preining: Software Development as mathematician in academia everyone bites the dust

Is it possible to do software development, mathematical or not, as mathematician in academics? This is a question I was asking myself recently a lot, seeing my own development from logician at a state university getting rid of foreigners to software developer. And then, a friend pointed me to this very depressing document: The origins of SageMath by William Stein, the main developer of SageMath. And I realized that it seems to be a global phenomenon that mathematicians who are interested in software development have to leave academics. What a sad affair. SageMath has a clear mission:
Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab.
All the Ma -software packages are commercial, and expensive. On the other hand they often have very good algorithms implemented. The Sage developers invested lots of time, energy, and brain power to develop excellent algorithm in an open source project for the mathematical researcher, but this investment wasn t honored in academic life. To quote from the presentation:
Issues with software dev in academia
  • Hard money for software development is virtually nonexistent: I can t think of anyone I know who got tenured based on his or her software.
  • Researchers on soft money are systematically discriminated against in favor of tenure-track and tenured faculty.
  • Researchers are increasingly evaluated solely on bibliometric counts rather than an informed assessment of their overall portfolio of papers, code, software, industry engagement, or student supervision.
The origins of SageMath, p.31
I can fully agree to this. Both from my own experience as well as from those around me. The presentation slides are full of other examples, from the developers of NumPy, Jupyter, as well as statements by Stephen Wolfram from Mathematica about this issue. A textbook how to not setup academia. My assumption was that this hits only on non-tenured staff, the academic precariat. It is shocking to see that even William Stein with a tenure position is leaving academics. It seems the times are not ready
Every great open source math library is built on the ashes of someone s academic career.
The origins of SageMath, p.32

25 July 2017

Norbert Preining: Debian/TeX Live 2017.20170724-1

Yesterday I uploaded the first update of the TeX Live packages in Debian after TeX Live 2017 has entered Debian/unstable. The packages should by now have reached most mirrors. Nothing spectacular here besides a lot of updates and new packages. If I have to pick one update it would be the one of algorithm2e, a package that has seen lots of use and some bugs due to two years of inactivity. Good to see a new release. Enjoy. New packages algolrevived, invoice2, jfmutil, maker, marginfit, pst-geometrictools, pst-rputover, pxufont, shobhika, tikzcodeblocks, zebra-goodies. Updated packages acmart, adobemapping, algorithm2e, arabluatex, archaeologie, babel, babel-french, bangorexam, beamer, beebe, biblatex-gb7714-2015, bibleref, br-lex, bxjscls, combofont, computational-complexity, dozenal, draftfigure, elzcards, embrac, esami, factura, fancyhdr, fei, fithesis, fmtcount, fontspec, fonttable, forest, fvextra, genealogytree, gotoh, GS1, l3build, l3experimental, l3kernel, l3packages, latexindent, limap, luapackageloader, lwarp, mcf2graph, microtype, minted, mptopdf, pdfpages, polynom, powerdot, probsoln, pxbase, pxchfon, pythontex, reledmac, siunitx, struktex, tcolorbox, tetex, texdirflatten, uowthesistitlepage, uptex-fonts, xcharter.

Next.