Enable fast recycling TIME-WAIT sockets. Default value is 0. It should not be changed without advice/request of technical experts.

Allow to reuse TIME-WAIT sockets for new connections when it is safe from protocol viewpoint. Default value is 0. It should not be changed without advice/request of technical experts.

Enable fast recycling of TIME-WAIT sockets. Enabling this option is not recommended since this causes problems when working with NAT (Network Address Translation).

About TIME-WAIT state Let s rewind a bit and have a close look at this TIME-WAIT state. What is it? See the TCP state diagram below²: Only the end closing the connection first will reach the TIME-WAIT state. The other end will follow a path which usually permits to quickly get rid of the connection. You can have a look at the current state of connections with ss -tan:

$ ss -tan   head -5
LISTEN     0  511             *:80              *:*     
SYN-RECV   0  0     192.0.2.145:80    203.0.113.5:35449
SYN-RECV   0  0     192.0.2.145:80   203.0.113.27:53599
ESTAB      0  0     192.0.2.145:80   203.0.113.27:33605
TIME-WAIT  0  0     192.0.2.145:80   203.0.113.47:50685

Purpose There are two purposes for the TIME-WAIT state:

• The most known one is to prevent delayed segments from one connection being accepted by a later connection relying on the same quadruplet (source address, source port, destination address, destination port). The sequence number also needs to be in a certain range to be accepted. This narrows a bit the problem but it still exists, especially on fast connections with large receive windows. RFC 1337 explains in details what happens when the TIME-WAIT state is deficient³. Here is an example of what could be avoided if the TIME-WAIT state wasn t shortened:

• The other purpose is to ensure the remote end has closed the connection. When the last ACK is lost, the remote end stays in the LAST-ACK state⁴. Without the TIME-WAIT state, a connection could be reopened while the remote end still thinks the previous connection is valid. When it receives a SYN segment (and the sequence number matches), it will answer with a RST as it is not expecting such a segment. The new connection will be aborted with an error:

RFC 793 requires the TIME-WAIT state to last twice the time of the MSL. On Linux, this duration is not tunable and is defined in include/net/tcp.h as one minute:

#define TCP_TIMEWAIT_LEN (60*HZ) /* how long to wait to destroy TIME-WAIT
                                  * state, about 60 seconds     */

There have been propositions to turn this into a tunable value but it has been refused on the ground the TIME-WAIT state is a good thing.

Problems Now, let s see why this state can be annoying on a server handling a lot of connections. There are three aspects of the problem:

• the slot taken in the connection table preventing new connections of the same kind,
• the memory occupied by the socket structure in the kernel, and
• the additional CPU usage.

The result of ss -tan state time-wait wc -l is not a problem per se!

Connection table slot A connection in the TIME-WAIT state is kept for one minute in the connection table. This means, another connection with the same quadruplet (source address, source port, destination address, destination port) cannot exist. For a web server, the destination address and the destination port are likely to be constant. If your web server is behind a L7 load-balancer, the source address will also be constant. On Linux, the client port is by default allocated in a port range of about 30,000 ports (this can be changed by tuning net.ipv4.ip_local_port_range). This means that only 30,000 connections can be established between the web server and the load-balancer every minute, so about 500 connections per second. If the TIME-WAIT sockets are on the client side, such a situation is easy to detect. The call to connect() will return EADDRNOTAVAIL and the application will log some error message about that. On the server side, this is more complex as there is no log and no counter to rely on. In doubt, you should just try to come with something sensible to list the number of used quadruplets:

$ ss -tan 'sport = :80'   awk ' print $(NF)" "$(NF-1) '   \
>     sed 's/:[^ ]*//g'   sort   uniq -c
    696 10.24.2.30 10.33.1.64
   1881 10.24.2.30 10.33.1.65
   5314 10.24.2.30 10.33.1.66
   5293 10.24.2.30 10.33.1.67
   3387 10.24.2.30 10.33.1.68
   2663 10.24.2.30 10.33.1.69
   1129 10.24.2.30 10.33.1.70
  10536 10.24.2.30 10.33.1.73

The solution is more quadruplets⁵. This can be done in several ways (in the order of difficulty to setup):

• use more client ports by setting net.ipv4.ip_local_port_range to a wider range,
• use more server ports by asking the web server to listen to several additional ports (81, 82, 83, ),
• use more client IP by configuring additional IP on the load balancer and use them in a round-robin fashion,
• use more server IP by configuring additional IP on the web server⁶.

Of course, a last solution is to tweak net.ipv4.tcp_tw_reuse and net.ipv4.tcp_tw_recycle. Don t do that yet, we will cover those settings later.

Memory With many connections to handle, leaving a socket open for one additional minute may cost your server some memory. For example, if you want to handle about 10,000 new connections per second, you will have about 600,000 sockets in the TIME-WAIT state. How much memory does it represent? Not that much! First, from the application point of view, a TIME-WAIT socket does not consume any memory: the socket has been closed. In the kernel, a TIME-WAIT socket is present in three structures (for three different purposes):

1. A hash table of connections, named the TCP established hash table (despite containing connections in other states) is used to locate an existing connection, for example when receiving a new segment. Each bucket of this hash table contains both a list of connections in the TIME-WAIT state and a list of regular active connections. The size of the hash table depends on the system memory and is printed at boot:

   $ dmesg   grep "TCP established hash table"
   [    0.169348] TCP established hash table entries: 65536 (order: 8, 1048576 bytes)
   

   It is possible to override it by specifying the number of entries on the kernel command line with the thash_entries parameter. Each element of the list of connections in the TIME-WAIT state is a struct tcp_timewait_sock, while the type for other states is struct tcp_sock⁷:

   struct tcp_timewait_sock  
       struct inet_timewait_sock tw_sk;
       u32    tw_rcv_nxt;
       u32    tw_snd_nxt;
       u32    tw_rcv_wnd;
       u32    tw_ts_offset;
       u32    tw_ts_recent;
       long   tw_ts_recent_stamp;
    ;
   struct inet_timewait_sock  
       struct sock_common  __tw_common;
       int                     tw_timeout;
       volatile unsigned char  tw_substate;
       unsigned char           tw_rcv_wscale;
       __be16 tw_sport;
       unsigned int tw_ipv6only     : 1,
                    tw_transparent  : 1,
                    tw_pad          : 6,
                    tw_tos          : 8,
                    tw_ipv6_offset  : 16;
       unsigned long            tw_ttd;
       struct inet_bind_bucket *tw_tb;
       struct hlist_node        tw_death_node;
    ;
   

2. A set of lists of connections, called the death row , is used to expire the connections in the TIME-WAIT state. They are ordered by how much time left before expiration. It uses the same memory space as for the entries in the hash table of connections. This is the struct hlist_node tw_death_node member of struct inet_timewait_sock.
3. A hash table of bound ports, holding the locally bound ports and the associated parameters, is used to determine if it is safe to listen to a given port or to find a free port in the case of dynamic bind. The size of this hash table is the same as the size of the hash table of connections:

   $ dmesg   grep "TCP bind hash table"
   [    0.169962] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
   

   Each element is a struct inet_bind_socket. There is one element for each locally bound port. A TIME-WAIT connection to a web server is locally bound to the port 80 and shares the same entry as its sibling TIME-WAIT connections. On the other hand, a connection to a remote service is locally bound to some random port and does not share its entry.

So, we are only concerned by the space occupied by struct tcp_timewait_sock and struct inet_bind_socket. There is one struct tcp_timewait_sock for each connection in the TIME-WAIT state, inbound or outbound. There is one dedicated struct inet_bind_socket for each outbound connection and none for an inbound connection. A struct tcp_timewait_sock is only 168 bytes while a struct inet_bind_socket is 48 bytes:

$ sudo apt-get install linux-image-$(uname -r)-dbg
[...]
$ gdb /usr/lib/debug/boot/vmlinux-$(uname -r)
(gdb) print sizeof(struct tcp_timewait_sock)
 $1 = 168
(gdb) print sizeof(struct tcp_sock)
 $2 = 1776
(gdb) print sizeof(struct inet_bind_bucket)
 $3 = 48

So, if you have about 40,000 inbound connections in the TIME-WAIT state, it should eat less than 10MB of memory. If you have about 40,000 outbound connections in the TIME-WAIT state, you need to account for 2.5MB of additional memory. Let s check that by looking at the output of slabtop. Here is the result on a server with about 50,000 connections in the TIME-WAIT state, 45,000 of which are outbound connections:

$ sudo slabtop -o   grep -E '(^  OBJS tw_sock_TCP tcp_bind_bucket)'
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME                   
 50955  49725  97%    0.25K   3397       15     13588K tw_sock_TCP            
 44840  36556  81%    0.06K    760       59      3040K tcp_bind_bucket

There is nothing to change here: the memory used by TIME-WAIT connections is really small. If your server need to handle thousands of new connections per second, you need far more memory to be able to efficiently push data to clients. The overhead of TIME-WAIT connections is negligible.

CPU On the CPU side, searching for a free local port can be a bit expensive. The work is done by the inet_csk_get_port() function which uses a lock and iterate on locally bound ports until a free port is found. A large number of entries in this hash table is usually not a problem if you have a lot of outbound connections in the TIME-WAIT state (like ephemeral connections to a memcached server): the connections usually share the same profile, the function will quickly find a free port as it iterates on them sequentially.

Other solutions If you still think you have a problem with TIME-WAIT connections after reading the previous section, there are three additional solutions to solve them:

• disable socket lingering,
• net.ipv4.tcp_tw_reuse, and
• net.ipv4.tcp_tw_recycle.

Socket lingering When close() is called, any remaining data in the kernel buffers will be sent in the background and the socket will eventually transition to the TIME-WAIT state. The application can continue to work immediatly and assume that all data will eventually be safely delivered. However, an application can choose to disable this behaviour, known as socket lingering. There are two flavors:

1. In the first one, any remaining data will be discarded and instead of closing the connection with the normal four-packet connection termination sequence, the connection will be closed with a RST (and therefore, the peer will detect an error) and will be immediatly destroyed. No TIME-WAIT state in this case.
2. With the second flavor, if there is any data still remaining in the socket send buffer, the process will sleep when calling close() until either all the data is sent and acknowledged by the peer or the configured linger timer expires. It is possible for a process to not sleep by setting the socket as non-blocking. In this case, the same process happens in the background. It permits the remaining data to be sent during a configured timeout but if the data is succesfully sent, the normal close sequence is run and you get a TIME-WAIT state. And on the other case, you ll get the connection close with a RST and the remaining data is discarded.

In both cases, disabling socket lingering is not a one-size-fits-all solution. It may be used by some applications like HAProxy or Nginx when it is safe to use from the upper protocol point of view. There are good reasons to not disable it unconditionnaly.

net.ipv4.tcp_tw_reuse The TIME-WAIT state prevents delayed segments to be accepted in an unrelated connection. However, on certain conditions, it is possible to assume a new connection s segment cannot be misinterpreted with an old connection s segment. RFC 1323 presents a set of TCP extensions to improve performance over high-bandwidth paths. Among other things, it defines a new TCP option carrying two four-byte timestamp fields. The first one is the current value of the timestamp clock of the TCP sending the option while the second one is the most recent timestamp received from the remote host. By enabling net.ipv4.tcp_tw_reuse, Linux will reuse an existing connection in the TIME-WAIT state for a new outgoing connection if the new timestamp is strictly bigger than the most recent timestamp recorded for the previous connection: an outgoing connection in the TIME-WAIT state can be reused after just one second. How is it safe? The first purpose of the TIME-WAIT state was to avoid duplicate segments to be accepted in an unrelated connection. Thanks to the use of timestamps, such a duplicate segments will come with an outdated timestamp and therefore be discarded. The second purpose was to ensure the remote end is not in the LAST-ACK state because of the lost of the last ACK. The remote end will retransmit the FIN segment until:

1. it gives up (and tear down the connection), or
2. it receives the ACK it is waiting (and tear down the connection), or
3. it receives a RST (and tear down the connection).

If the FIN segments are received in a timely manner, the local end socket will still be in the TIME-WAIT state and the expected ACK segments will be sent. Once a new connection replaces the TIME-WAIT entry, the SYN segment of the new connection is ignored (thanks to the timestamps) and won t be answered by a RST but only by a retransmission of the FIN segment. The FIN segment will then be answered with a RST (because the local connection is in the SYN-SENT state) which will allow the transition out of the LAST-ACK state. The initial SYN segment will eventually be resent (after one second) because there was no answer and the connection will be established without apparent error, except a slight delay: It should be noted that when a connection is reused, the TWRecycled counter is increased (despite its name).

net.ipv4.tcp_tw_recycle This mechanism also relies on the timestamp option but affects both incoming and outgoing connections which is handy when the server usually closes the connection first⁸. The TIME-WAIT state is scheduled to expire sooner: it will be removed after the retransmission timeout (RTO) interval which is computed from the RTT and its variance. You can spot the appropriate values for a living connection with the ss command:

$ ss --info  sport = :2112 dport = :4057
State      Recv-Q Send-Q    Local Address:Port        Peer Address:Port   
ESTAB      0      1831936   10.47.0.113:2112          10.65.1.42:4057    
         cubic wscale:7,7 rto:564 rtt:352.5/4 ato:40 cwnd:386 ssthresh:200 send 4.5Mbps rcv_space:5792

To keep the same guarantees the TIME-WAIT state was providing, while reducing the expiration timer, when a connection enters the TIME-WAIT state, the latest timestamp is remembered in a dedicated structure containing various metrics for previous known destinations. Then, Linux will drop any segment from the remote host whose timestamp is not strictly bigger than the latest recorded timestamp, unless the TIME-WAIT state would have expired:

if (tmp_opt.saw_tstamp &&
    tcp_death_row.sysctl_tw_recycle &&
    (dst = inet_csk_route_req(sk, &fl4, req, want_cookie)) != NULL &&
    fl4.daddr == saddr &&
    (peer = rt_get_peer((struct rtable *)dst, fl4.daddr)) != NULL)  
        inet_peer_refcheck(peer);
        if ((u32)get_seconds() - peer->tcp_ts_stamp < TCP_PAWS_MSL &&
            (s32)(peer->tcp_ts - req->ts_recent) >
                                        TCP_PAWS_WINDOW)  
                NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_PAWSPASSIVEREJECTED);
                goto drop_and_release;
         
 

When the remote host is in fact a NAT device, the condition on timestamps will forbid allof the hosts except one behind the NAT device to connect during one minute because they do not share the same timestamp clock. In doubt, this is far better to disable this option since it leads to difficult to detect and difficult to diagnose problems. The LAST-ACK state is handled in the exact same way as for net.ipv4.tcp_tw_recycle.

Summary The universal solution is to increase the number of possible quadruplets by using, for example, more server ports. This will allow you to not exhaust the possible connections with TIME-WAIT entries. On the server side, do not enable net.ipv4.tcp_tw_recycle unless you are pretty sure you will never have NAT devices in the mix. Enabling net.ipv4.tcp_tw_reuse is useless for incoming connections. On the client side, enabling net.ipv4.tcp_tw_reuse is another almost-safe solution. Enabling net.ipv4.tcp_tw_recycle in addition to net.ipv4.tcp_tw_reuse is mostly useless. And a final quote by W. Richard Stevens, in Unix Network Programming:

The TIME_WAIT state is our friend and is there to help us (i.e., to let old duplicate segments expire in the network). Instead of trying to avoid the state, we should understand it.

---

1. Notably, fiddling with net.netfilter.nf_conntrack_tcp_timeout_time_wait won t change anything on how the TCP stack will handle the TIME-WAIT state.
2. This diagram is licensed under the LaTeX Project Public License 1.3. The original file is available on this page.
3. The first work-around proposed in RFC 1337 is to ignore RST segments in the TIME-WAIT state. This behaviour is controlled by net.ipv4.rfc1337 which is not enabled by default on Linux because this is not a complete solution to the problem described in the RFC.
4. While in the LAST-ACK state, a connection will retransmit the last FIN segment until it gets the expected ACK segment. Therfore, it is unlikely we stay long in this state.
5. On the client side, older kernels also have to find a free local tuple (source address and source port) for each outgoing connection. Increasing the number of server ports or IP won t help in this case. Linux 3.2 is recent enough to be able to share the same local tuple for different destinations. Thanks to Willy Tarreau for his insight on this aspect.
6. This last solution may seem a bit dumb since you could just use more ports but some servers are not able to be configured this way. The before last solution can also be quite cumbersome to setup, depending on the load-balancing software, but uses less IP than the last solution.
7. The use of a dedicated memory structure for sockets in the TIME-WAIT is here since Linux 2.6.14. The struct sock_common structure is a bit more verbose and I won t copy it here.
8. When the server closes the connection first, it gets the TIME-WAIT state while the client will consider the corresponding quadruplet free and hence may reuse it for a new connection.

You don t see me complaining that everything on the web is in English, do you?

Using the CuBox

On the i2 and i4 models, you can use it either with a USB keyboard and a HDMI monitor, or by a serial consoles (smaller models do not have a serial console). I don't have a HDMI monitor handy (only a projector), so I prefer to use the serial terminal. Important details to avoid frustration: The USB keyboard has to be connected to the lower USB port, or it will be ignored during the boot process. And make sure your serial terminal is configured not to use hardware flow control. Minicom is configured by default to use hardware flow control, so it was not sending any characters to the CuBox. ^A-O gets you to the Minicom configuration, select Serial port setup, and disable it.

Set up the SD card

I created a 2GB partition, but much less can suffice; I'd leave it at least to 1GB to do the base install, although it can be less once the system is set up (more on this later). Partition and format using your usual tools (fdisk+mke2fs, or gparted, or whatever suits your style).

Install the bootloader

I followed up the instructions on this CuBox-i forums thread to get the SPL and uBoot bootloader running. In short, from this Google Drive folder, download the SPL-U-Boot.img.xz file, uncompress it (xz --decompress SPL-U-Boot.img.xz), and write it to the SD card just after the partition map: As root,
# dd if=SPL-U-Boot.img of=/dev/mmcblk0 bs=1024 seek=1.
Actually, to be honest: As I wanted something basic to be able to debug from, I downloaded (from the same Google Drive) the busybox.img.gz file. That's a bit easier to install from: xz --decompress busybox.img.xz, and just dump it into the SD from the beginning (as it does already include a partition table):
# dd if=busybox.img of=/dev/mmcblk0
This card is already bootable and minimal, and allows to debug some bits from the CuBox-i itself (as we will see shortly).
After this step, I created a second partition, as I said earlier. So, my mmcblk0p1 partition holds Busybox, and the second will hold Debian. We are still working from the x86 system, so we mount the SD card in /media/mmcblk0p2

Installing the base system

Without debian-installer to do the heavy lifting, I went for debootstrap. As I ran it from my PC, debootstrap's role will be for this first stage only to download and do a very initial pre-unpacking of the files: Bootstrapping a foreign architecture implies, right, using the --foreign switch:
debootstrap --foreign --arch=armhf wheezy /media/mmcblk0p2 http://http.debian.net/debian
You can add some packages you often use by specifying --include=foo,bar,baz
So, take note notes: This board is capable of running the armhf architecture (HF for Hardware Float). It can also run armel, but I understand it is way slower.

First boot (with busybox)

So, once debootstrap finishes, you are good to go to the real hardware! Unmount the SD card, put it in the little guy, plug your favorite console in (I'm using the serial port), and plug the power in! You should immediately see something like:

U-Boot SPL 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
Boot Device: SD1                                              
reading u-boot.img
Load image from RAW...
 
 
U-Boot 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
 
CPU:   Freescale i.MX6Q rev1.2 at 792 MHz
Reset cause: POR                         
Board: MX6-CuBox-i
DRAM:  2 GiB      
MMC:   FSL_SDHC: 0
In:    serial     
Out:   vga   
Err:   vga
Net:   phydev = 0x0
Phy not found      
PHY reset timed out
FEC                
(Re)start USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
       scanning usb for ethernet devices... 0 Ethernet Device(s) found
Hit any key to stop autoboot:  3

Let it boot (that means, don't stop autoboot), and you will soon see a familiar #, showing you are root in the busybox environment. Great! Now, mount the Debian partition:
# mount /dev/mmcblk0p2 /mnt

Finishing debootstrap's task

With everything in place, it's time for debootstrap to work. Chroot into the Debian partition:
# chroot /mnt
And ask Debootstrap to finish what it started:
# debootstrap --second-stage
Be patient, as this step takes quite a bit to be finished.

Some extra touches...

After this is done, your Debian system is almost ready to be booted into. Why almost? Because it still does not have any users, does not know its own name nor knows I want to use it via a serial terminal. Three very simple tasks to fix. First two:

# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
# echo cubox-i.gwolf.org > /etc/hostname

For the second one, add a line to /etc/inittab specifying the details of the serial console. You can just do this:
# echo 'T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt100' >> /etc/inittab

Boot into Debian!

So, ready to boot Debian? Ok, first exit the chroot shell, to go back to the Busybox shell, unmount the Debian partition, and set the root partition read-only:

# exit
# umount /mnt
# mount / -o remount,ro

Disconnect and connect power, and now, do interrupt the boot process when you see the Hit any key to stop automount prompt. To see the configuration of uboot, you can type printenv We will only modify the parameters given to the kernel:

CuBox-i U-Boot > setenv root /dev/mmcblk0p2 rootfstype=ext3 ro rootwait
CuBox-i U-Boot > boot

So, the kernel will load, and a minimal Debian system will be initialized. In my case, I get the following output:

** File not found /boot/busyEnv.txt **
4703740 bytes read in 390 ms (11.5 MiB/s)
## Booting kernel from Legacy Image at 10000000 ...
   Image Name:   Linux-3.0.35-8                    
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    4703676 Bytes = 4.5 MiB              
   Load Address: 10008000               
   Entry Point:  10008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
 
Starting kernel ...
 
Unable to get enet.0 clock
pwm-backlight pwm-backlight.0: unable to request PWM for backlight
pwm-backlight pwm-backlight.1: unable to request PWM for backlight
_regulator_get: get() with no identifier
mxc_sdc_fb mxc_sdc_fb.2: NO mxc display driver found!
INIT: version 2.88 booting
[info] Using makefile-style concurrent boot in runlevel S.
[....] Starting the hotplug events dispatcher: udevd. ok 
[....] Synthesizing the initial hotplug events...done.
[....] Waiting for /dev to be fully populated...done.
[....] Activating swap...done.
[....] Cleaning up temporary files... /tmp. ok 
[....] Activating lvm and md swap...done.
[....] Checking file systems...fsck from util-linux 2.20.1
done.
[....] Mounting local filesystems...done.
[....] Activating swapfile swap...done.
[....] Cleaning up temporary files.... ok 
[....] Setting kernel variables ...done.
[....] Configuring network interfaces...done.
[....] Cleaning up temporary files.... ok 
[....] Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix. ok 
INIT: Entering runlevel: 2
[info] Using makefile-style concurrent boot in runlevel 2.
[....] Starting enhanced syslogd: rsyslogd. ok 
[....] Starting periodic command scheduler: cron. ok 
 
Debian GNU/Linux 7 cubox-i.gwolf.org ttymxc0
 
cubox-i login: 

And that's it, the system is live and ready for my commands!

root@cubox-i:~# apt-get clean
root@cubox-i:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs         1008M  347M  611M  37% /
/dev/root      1008M  347M  611M  37% /
devtmpfs        881M     0  881M   0% /dev
tmpfs           177M  132K  177M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           353M     0  353M   0% /run/shm

• What Are You Afraid Of: Most people would have probably gone with their first hit for the topic, but I think this song transports the point extremely well too: It's just a color and I'm color blind, the only color I know is the color of my mind. There's only one race and that's the human race, and every human being's got the right to feel save.
• The Faggot In You: Well, it's like Coyote Too twittered or Andy Singer drew it. Get over it and think about what actually stirs these emotions in you. If you're so sure and you feel secure about yourself and your reality, then why do you need reject and refuse where other people stand sexually?
• Life Will Kill You: Let's face the fact, it will. ... and given the choice between your own life and death I suggest that you cherish the time you have left 'cause time waits for noone and we're all growing older. Life for today, not for a future that might never come.

/music permanent link Comments: 4 Flattr this

pass is a very simple password store that keeps passwords inside gpg2(1) encrypted files inside a simple directory tree residing at ~/.password-store. The pass utility provides a series of commands for manipulating the password store, allowing the user to add, remove, edit, synchronize, generate, and manipulate passwords.

yum/apt-get install pass

pass init GPGKEYID

pass generate sudo_password 30

pass generate gmail_password 30

pass ls

Password Store
  gmail_password
  sudo_password
  root_password

pass edit password_name

pass rm password_name

pass -c password_name

E:Failed to mount /efs (Invalid Argument)

1. Using heimdall, flash the TeamWin recovery onto your phone (drop into download mode first hold VolDown+Home+Power):

   heimdall flash --recovery twrp.img
   

2. Boot into recovery (VolUp+Home+Power), select Advanced -> Terminal , and take an image of the EFS partition onto the external SD card you should have already in the phone:

   dd if=/dev/block/mmcblk0p3 of=/external_sd/efs.img
   

3. Shutdown the phone, mount the SD card on your computer, then turn your EFS partition image into a loopback device and fsck it:

   sudo losetup -f .../efs.img
   sudo fsck -f /dev/loop0
   

   With a bit of luck, the partition won t be a complete write-off and you ll be able to salvage the contents of the files, if not the exact filesystem structure. Incidentally, if the filesystem was completely stuffed, you could get someone else s EFS partition and change the IMEI and MAC addresses and you d probably be golden, but that would quite possibly be illegal or something, so don t do that.
4. Now comes the fun part putting the filesystem back together. After fscking, mount the image somewhere on your computer:

   mount /dev/loop0 /mnt
   

   In my case, I had about a dozen files living in lost+found, and I figured that wasn t a positive outcome. I did try, just in case, writing the fsck d filesystem image back to the phone, in the hope that it just needed to mount the filesystem to boot, but no dice. Instead, I had to find out where these lost soul^Wfiles were supposed to live. Luckily, a colleague of mine also has an S3 (the ever-so-slightly-different GT-I9300T), and he was kind enough to let me take a copy of his EFS partition, and use that as a file location template. Using a combination of file sizes, permissions/ownerships, and inode numbers (I knew the -i option to ls would come in handy someday!), I was able to put all the lost files back where they should be.
5. Unmount all those EFS filesystems, losetup -d /dev/loop0, and put the fixed up EFS partition image back onto your SD card for the return trip to the phone.
6. Now, with a filesystem image that looks reasonable, it s time to write it back onto the phone and see what happens. Copy it onto the SD card, boot up into recovery again, get a shell, and a bit more dd:

   dd if=/external_sd/efs.img of=/dev/block/mmcblk0p3
   

7. With a bit of luck, your phone may just boot back up now. In my case, I d done so many other things to my phone trying to get it back up and running (including flashing custom ROMs and what have you) that I needed to flash Cyanogen, boot it, and wait at the boot screen for about 15 minutes (I shit you not, 15 minutes of Gah is my phone going to work?!? ) before it came up and lo! I had a working phone again. And about 27 SMSes. Sigh, back to work

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Authentication

Each participant should be able to know specifically who the other parties are on the chat.

Confidentiality

The content of the messages should only be intelligible to the parties involved with the chat; it should appear opaque or encrypted to anyone else listening in. Note that confidentiality effectively depends on authentication -- if you don't know who you're talking to, you can't make sensible assertions about confidentiality.

Forward secrecy

Assuming the parties communicating are operating in good faith, forward secrecy offers protection against a special kind of adversary: one who logs the encrypted chat, and subsequently steals either party's long-term secret key. Without forward secrecy, such an adversary would be able to discover the content of the messages, violating the confidentiality characteristic. With forward secrecy, this adversary is be stymied and the messages remain confidential.

Deniability

Deniability only comes into play when one of the parties is no longer operating in good faith (e.g. their computer is compromised, or they are collaborating with an adversary). In this context, if Alice is chatting with Bob, she does not want Bob to be able to cryptographically prove to anyone else that she made any of the specific statements in the conversation. This is the focus of Monday's discussion. To be clear, this kind of deniability means Alice can correctly say "you have no cryptographic proof I said X", but it does not let her assert "here is cryptographic proof that I did not say X" (I can't think of any protocol that offers the latter assertion). The opposite of deniability is a cryptographic proof of origin, which usually runs something like "only someone with access to Alice's secret key could have said X."

• laws themselves are not always just (e.g. consider chat communications between an interracial couple in the USA before 1967, if instant messaging had existed at the time), and
• law enforcement (or a legal adversary in civil litigation) may have a different understanding or interpretation of the law than you do (e.g. consider chat communications between a corporate or government whistleblower and a journalist).

Scripting SNMP An SNMP agent can deliver a lot of interesting information:

• interface counters with IF-MIB,
• IP addresses with IP-MIB,
• routing tables with IP-FORWARD-MIB,
• inventory with ENTITY-MIB,
• neighbors with LLDP-MIB.

You can gather those information manually with tools like snmpget and snmpwalk:

$ snmpwalk -v 2c -c public localhost IF-MIB::ifDescr      
IF-MIB::ifDescr.1 = STRING: lo
IF-MIB::ifDescr.2 = STRING: eth0
IF-MIB::ifDescr.3 = STRING: eth1

However, building robust scripts with them is quite challenging. For example, if you wanted to get the descriptions of all active interfaces as well as the total number of octets transmitted, you could do something like that:

#!/bin/sh
set -e
host="$ 1:-localhost "
community="$ 2:-public "
args="-v2c -c $community $host"
for idx in $(snmpwalk -Ov -OQ $args IF-MIB::ifIndex); do
    descr=$(snmpget -Ov -OQ $args IF-MIB::ifDescr.$idx)
    oper=$(snmpget -Ov -OQ $args IF-MIB::ifOperStatus.$idx)
    in=$(snmpget -Ov -OQ $args IF-MIB::ifInOctets.$idx)
    out=$(snmpget -Ov -OQ $args IF-MIB::ifOutOctets.$idx)
    [ x"$descr" != x"lo" ]   continue
    [ x"$oper" = x"up" ]   continue
    echo $descr $in $out
done

Hopefully, SNMP bindings in various languages are pretty common. For example, Net-SNMP ships with a Python binding:

import argparse
import netsnmp
parser = argparse.ArgumentParser()
parser.add_argument("host", default="localhost", nargs="?",
                    help="Agent to retrieve variables from")
parser.add_argument("community", default="public", nargs="?",
                    help="Community to query the agent")
options = parser.parse_args()
args =  
    "Version": 2,
    "DestHost": options.host,
    "Community": options.community
 
for idx in netsnmp.snmpwalk(netsnmp.Varbind("IF-MIB::ifIndex"),
                            **args):
    descr, oper, cin, cout = netsnmp.snmpget(
        netsnmp.Varbind("IF-MIB::ifDescr", idx),
        netsnmp.Varbind("IF-MIB::ifOperStatus", idx),
        netsnmp.Varbind("IF-MIB::ifInOctets", idx),
        netsnmp.Varbind("IF-MIB::ifOutOctets", idx),
        **args)
    assert(descr is not None and
           cin is not None and
           cout is not None) #  
    if descr == "lo":
        continue
    if oper != "1": #  
        continue
    print("     ".format(descr, cin, cout))

This binding is quite primitive and has several drawbacks:

1. It exports everything as strings. See .
2. Error handling is just deficient. If you mispell something, like a variable name, you ll get snmp_build: unknown failure on the standard error. No exception. If a variable does not exist, you ll get None instead. See .

This inability to sanely handle failures makes this binding quite dangerous to use in scripts. Imagine making important modifications on the basis of the returned values. If you forget to check against None, your script may cause havoc!

Snimpy Because I didn t find any reliable Python binding for SNMP, I decided to write Snimpy with two goals in mind:

1. Leverage information contained in MIBs to provide a pythonic interface.
2. Any error condition should raise an exception.

Here is how the previous script could be written:

#!/usr/bin/env snimpy
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("host", default="localhost", nargs="?",
                    help="Agent to retrieve variables from")
parser.add_argument("community", default="public", nargs="?",
                    help="Community to query the agent")
options = parser.parse_args()
m = M(options.host, options.community, 2)
load("IF-MIB")
for idx in m.ifDescr:
    if m.ifDescr[idx] == "lo":
        continue
    if m.ifOperStatus[idx] != "up":
        continue
    print("     ".format(m.ifDescr[idx],
                            m.ifInOctets[idx],
                            m.ifOutOctets[idx]))

You can also use a list comprehension:

load("IF-MIB")
print("\n".join([ "     ".format(m.ifDescr[idx],
                                    m.ifInOctets[idx],
                                    m.ifOutOctets[idx])
                  for idx in m.ifDescr
                  if m.ifDescr[idx] != "lo"
                  and m.ifOperStatus[idx] == "up" ]))

Here is another simple example to get the routing database from the agent:

load("IP-FORWARD-MIB")
m=M("localhost", "public", 2)
routes = m.ipCidrRouteNextHop
for x in routes:
    net, netmask, tos, src = x
    print(" :>15s / :<15s  via  :<15s  src  :<15s ".format(
        net, netmask, routes[x], src))

IP-FORWARD-MIB::ipCidrRouteNextHop is a more complex table with a compound index. Despite this, querying the table still seems natural. Have a look at Snimpy s documentation for more information. Behind the hood, SNMP requests are handled by PySNMP and MIB parsing is done with libsmi². Snimpy supports both Python 2, Python 3 and Pypy.

---

1. A MIB is defined using a subset of ASN.1 called SMI. However, it is not uncommon to refer to the definition as a MIB too.
2. Unfortunately, there is currently no robust SMI parser written in pure Python. For example, PySNMP relies on smidump which comes with libsmi. Snimpy uses a custom CFFI wrapper around libsmi.