wondershaper is a real wonder

Have you ever experienced "slow internet" when there is another ongoing transfer on your wire which fully saturates the bandwidth. Even if one can accept high latency of HTTP transfers (sooner or later the page will appear in the browser), in case of interactive sessions like SSH it is completely unacceptable. Wondershaper is a great remedy with minimal side effects - especially if your are running your router on Linux machine.

Wondershaper is using CBQ instead of HTB and I am aware that there are better approaches to traffic shaping in Linux. But "traffic shaping" is usually associated with the context of splitting the bandwidth of single internet link among many hosts. This is too much in my case and wondershaper does exactly what I expect.

My ADSL line is 1024/256kb officially. But in logs I can see different values:

ATM dev 0: ADSL line is up (1312 kb/s down | 320 kb/s up)

At first I tried to provide these values to wondershaper, however they were too high (high latency still and packet drops in addition). By using different values I finally established that official numbers give the best trade-off between latency and link speed.

In case of my link the average download rate is around 126KB/s. Now with wondershaper it is around 120KB/s - about 95% of the original speed - this the cost. However SSH sessions work like a charm when updates are being downloaded and the browser loads pages almost as fast as usually.

Now specific section of my /etc/network/interfaces (Debian specific) looks like:

auto ppp0
iface ppp0 inet ppp
    pre-up /usr/local/sbin/firewall.sh start
    pre-up while ! grep 'Line up' /proc/net/atm/speedtch:0 &>/dev/null; do sleep 1; done
    post-up echo "1" >/proc/sys/net/ipv4/ip_forward
    post-up wondershaper ppp0 1024 250
    pre-down wondershaper clean ppp0
    pre-down echo "0" >/proc/sys/net/ipv4/ip_forward
    post-down /usr/local/sbin/firewall.sh stop

Update

With the 1024 256 values I still had some latency where my uplink was fully saturated. The 1024 250 values seems to work OK.

smartmontools: debian lenny backport for armel architecture

I had to backport smartmontools. You can download it from xemantic-builds

dropbear: scp does not work, here is poor man's replacement

The dropbear project provides very interesting implementation of SSH server. It has advantage of very low memory footprint, which might be crucial in case of embedded devices. Personally I use dropbear on NSLU2 device (aka SLUG).

I was a little bit disappointed when I tried scp command against dropbear server (on debian host):

$ scp .ssh/id_rsa.pub root@slug:authorized_keys
bash: scp: command not found
lost connection

Another try:

sftp root@slug
Connecting to slug...
bash: /usr/lib/sftp-server: No such file or directory
Connection closed

Dropbear comes without scp and sftp-server commands. On debian you can find them in openssh-server package. Installing it would bring a lot of unwanted dependencies not even mentioning that I would have to make sure that OpenSSH server is not going to start and compete with my little dropbear. Here is simple scp-like solution.

cat .ssh/id_rsa.pub | ssh root@slug 'tee authorized_keys >/dev/null'

I hope it will help someone.

Nexus on debian

Nexus is very cool Maven Repository Manager, and debian is as cool server OS platform. I tried to match them. There was some pain, however finally I succeeded :).

Nexus is bundled as war now, thus the description is about installing it on tomcat.

In case of using debian etch (current stable), you may consider installing jdk 1.6 from backports. With jdk 1.6 you have to specify new JAVA_HOME=/usr/lib/jvm/java-6-sun in /etc/default/tomcat5.5.

Debian's tomcat has security manager turned on by default. Of course it could be easily disabled in /etc/default/tomcat5.5, however I feel safer when it is turned on :). Establishing security policy is not easy. My solution to the problem is based on work described in Mark Petrovic's article. I tweaked it a bit not to generate redundant rules. In case of nexus the resulting file looks as follows:

grant codeBase "file:${catalina.home}/bin/tomcat-juli.jar" {
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/WEB-INF/classes/logging.properties", "read";
};

grant codeBase "file:${catalina.base}/webapps/nexus/WEB-INF/lib/-" {
    permission java.util.PropertyPermission "*", "read,write";
    permission java.io.FilePermission "/", "read";
    permission java.io.FilePermission "${catalina.base}/logs", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.properties", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.xml", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/WEB-INF/log4j.properties", "read";
    permission java.io.FilePermission "${catalina.base}/temp", "read,write";
    permission java.io.FilePermission "${catalina.base}/temp/-", "read,write,delete";
    permission java.lang.RuntimePermission "defineClassInPackage.java.lang", "";
    permission java.lang.RuntimePermission "createClassLoader", "";
    permission java.lang.RuntimePermission "setContextClassLoader", "";
    permission java.lang.RuntimePermission "accessDeclaredMembers", "";
    permission java.lang.RuntimePermission "getenv.*", "";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.misc", "";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.reflect", "";
    permission java.lang.RuntimePermission "reflectionFactoryAccess", "";
    permission java.lang.RuntimePermission "getClassLoader", "";
    permission java.lang.RuntimePermission "modifyThread", "";
    permission java.io.FilePermission "${catalina.home}/sonatype-work", "read,write";
    permission java.io.FilePermission "${catalina.home}/sonatype-work/-", "read,write,delete";
    permission java.net.SocketPermission "*", "connect,resolve";
    permission java.lang.reflect.ReflectPermission "suppressAccessChecks", "";
    permission java.util.logging.LoggingPermission "control", "";
};

grant codeBase "file:${catalina.base}/webapps/nexus/WEB-INF/classes/-" {
    permission java.util.PropertyPermission "*", "read,write";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.properties", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.xml", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/WEB-INF/log4j.properties", "read";
    permission java.io.FilePermission "${catalina.home}/sonatype-work", "read,write";
    permission java.io.FilePermission "${catalina.home}/sonatype-work/-", "read,write,delete";
    permission java.lang.RuntimePermission "getenv.*", "";
    permission java.lang.RuntimePermission "defineClassInPackage.java.lang", "";
    permission java.lang.RuntimePermission "createClassLoader", "";
    permission java.lang.RuntimePermission "setContextClassLoader", "";
    permission java.lang.RuntimePermission "accessDeclaredMembers", "";
    permission java.lang.RuntimePermission "modifyThread", "";
    permission java.util.logging.LoggingPermission "control", "";
};

Copy these rules into 60nexus.policy file and place it in /etc/default/tomcat5.5. Restart of JVM is required to make these rules effective.

The line:

permission java.io.FilePermission "/", "read";

is redundant with other rules. The need of reading the whole filesystem seems to be a security flaw. I hope it will be eliminated in future nexus releases. I will fill the bug for this.

These security rules should work ok not only on debian, but also on any other system where tomcat is deployed. There are two system properties in use: catalina.home and catalina.base. This seems to be debian specific. In case of other systems catalina.home should be enough.

Nexus would create sonatype-work directory in user.home which is /usr/share/tomcat5.5 in case of debian. However with security manager we have to create it by ourselves.

host# mkdir /var/opt/sonatype-work
host# chown tomcat55:adm /var/opt/sonatype-work
host# ln -s /var/opt/sonatype-work /usr/share/tomcat5.5

The /var/opt/sonatype-work will become storage which should be backed up carefully when nexus is used not only as proxy, but also as local maven repository.

Now we can download the latest nexus from http://nexus.sonatype.org/using/download.html

Create logging.properties file with the following contents:

org.apache.juli.FileHandler.level = WARNING
java.util.logging.ConsoleHandler.level = WARNING

and put it into WEB-INF/classes inside the war.

Without this modification nexus would log every single HTTP request on console which would be redirected to catalina.out log file. To much verbose default logging seems to be another nexus bug.

Strip the version number from war file, stop tomcat with:

host# /etc/init.d/tomcat5.5 stop

Put nexus.war in /var/lib/tomcat5.5/webapps.

host# /etc/init.d/tomcat5.5 start

Nexus should be visible at http://debianhost:8180/nexus/

Upgradges should be as easy as putting new nexus war in webapps.