smartmontools: debian lenny backport for armel architecture

I had to backport smartmontools. You can download it from xemantic-builds

dropbear: scp does not work, here is poor man's replacement

The dropbear project provides very interesting implementation of SSH server. It has advantage of very low memory footprint, which might be crucial in case of embedded devices. Personally I use dropbear on NSLU2 device (aka SLUG).

I was a little bit disappointed when I tried scp command against dropbear server (on debian host):

$ scp .ssh/id_rsa.pub root@slug:authorized_keys
bash: scp: command not found
lost connection

Another try:

sftp root@slug
Connecting to slug...
bash: /usr/lib/sftp-server: No such file or directory
Connection closed

Dropbear comes without scp and sftp-server commands. On debian you can find them in openssh-server package. Installing it would bring a lot of unwanted dependencies not even mentioning that I would have to make sure that OpenSSH server is not going to start and compete with my little dropbear. Here is simple scp-like solution.

cat .ssh/id_rsa.pub | ssh root@slug 'tee authorized_keys >/dev/null'

I hope it will help someone.

Sparse image bundle for Time Machine backup

The command is:

$ hdiutil create -size 150g -fs HFS+J -nospotlight -imagekey sparse-band-size=131072 -volname “hostname-backup” /tmp/HOSTNAME_MACADDRESSWITHOUTCOLONS.sparsebundle

-size

the maximal size of the bundle in gigabytes (created image will be initially smaller though - about 250MB)

-fs HFS+J

the filesystem - using HFS is the main reason for preparing image anyway. TimeMachine depends on some HFS magic.

-nospotlight

prevents Mac OS from indexing this image

-imagekey sparse-band-size=131072

some sources on the Internet claim this is the best choice in terms of performance

When the image is ready, it should be moved to the network share where backup will be stored. Choosing the new location in TimeMachine preferences is the last step.

gwt-servlet-war project, no more annoying gwt-servlet in the classpath

All the GWT libraries are available in the central maven repositories now. There are 4 different jars for single platform (gwt-user, gwt-dev, gwt-dev with platform classifier (thus there is even more jars the 4) and gwt-servlet.

If you are developing gwt application which will be packaged as war, these libraries should be included in pom.xml with something like this:

 <dependencies>
   <dependency>
     <groupId>com.google.gwt</groupId>
     <artifactId>gwt-user</artifactId>
     <version>${gwtVersion}</version>
     <scope>provided</scope>
   </dependency>
   <dependency>
     <groupId>com.google.gwt</groupId>
     <artifactId>gwt-dev</artifactId>
     <version>${gwtVersion}</version>
     <classifier>${platform}-libs</classifier>
     <type>zip</type>
     <scope>provided</scope>
   </dependency>
   <dependency>
     <groupId>com.google.gwt</groupId>
     <artifactId>gwt-dev</artifactId>
     <version>${gwtVersion}</version>
     <classifier>${platform}</classifier>
     <scope>provided</scope>
   </dependency>
   <dependency>
     <groupId>com.google.gwt</groupId>
     <artifactId>gwt-servlet</artifactId>
     <version>${gwtVersion}</version>
   </dependency>
 </dependencies>

GWT jars marked with provided will be available during development, but not packaged into war. This approach have only one drawback, core gwt classes are present in both - gwt-user, and gwt-servlet jar in the same time. In case of developing project in eclipse with m2eclipse plugin it leads to very annoying behavior (not finding sources, showing two candidate types with the same name, etc.).

I have created new gwt-servlet-war project to overcome this problem. The idea is very simple. Instead of the last gwt-servlet dependency from above example, one should put:

   <dependency>
     <groupId>pl.ncdc.gwt</groupId>
     <artifactId>gwt-servlet-war</artifactId>
     <version>${gwtVersion}</version>
     <type>war</type>
   </dependency>

After this change, war packaging will be supported by so called "war overlays" where wars specified as dependencies are merged into war being built.

And guess what gwt-servlet-war contains? Only gwt-servlet library in WEB-INF/lib. Thus project is in fact single pom.xml. Anyway It have to be packaged according to gwt versioning scheme.

If anyone have any idea how to set up maven repository on top of code.google.com's subversion, please contribute it to the project. Maybe I should promote this jar to be put in central maven repo, or rather suggest people responsible for putting gwt artifacts there to provide something of this kind along the line?

Maven google code upload plugin

The project is located here.

Example setup (works well with my projects) looks like:

<plugin>
  <groupId>org.riedelcastro</groupId>
  <artifactId>maven-gcupload-plugin</artifactId>
  <version>1.0</version>
  <configuration>
    <uploads>
      <upload>
        <extensions>jar</extensions>
        <labels>Featured,Type-Archive,OpSys-All</labels>  
      </upload>
      <upload>
        <extensions>jar</extensions>
        <postfix>sources</postfix>
        <labels>Featured,Type-Source,OpSys-All</labels>
      </upload>
      <upload>
        <extensions>jar</extensions>
        <postfix>javadoc</postfix>
        <labels>Featured,Type-Docs,OpSys-All</labels>
      </upload>
    </uploads>
  </configuration>
</plugin>

The gcupload:gcupload maven target is added to our Hudson. We have differentia-javaica release build which takes care of everything - deploying jars to our Nexus as well as putting them to Google Code.

Differentia Javaica

I started a new project called differentia-javaica. Here is the quote from the original description:

The aim of this project is to compare two java source codes and check if they are equal. It is not a simple diff. It uses ANTLR to construct two Abstract Source Trees for java types and eventually compare these trees. As a consequence white spaces and comments will not affect comparison. Reordering of elements in source code will be treated as difference though.

This kind of comparison is especially helpful when writing unit tests for java source code generators. When we have expected source code it is possible to check if it equals to generated source code.

We are writing some Java source code generators right now at NCDC and this tools is quite helpful in unit testing. Thus we want to share it with community.

Even more X-mas cards

Source: eatliver.com

X-mas cards

XML find and replace - xmlsed how to

Sed is a great tool when it comes to string replacement. With simple command like:

$ sed -i -e 's/log4j\.rootLogger=debug/log4j\.rootLogger=warn/' log4j.properties

, it will change appropriate value in log4j.properties file. It is especially useful in automatic scripts which customize configuration.

The problem I have encountered lately was connected with using sed for string replacement in xml files. I tried different regular expressions, and multi-line matching, and it was a real pain. I needed a kind of "xmlsed" in fact. Then I realized that even I don't know sed script syntax good enough, I know language which is tailored at manipulation of XML, which is XSLT. :)

For example when we have xml log4j configuration like this:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
  <appender name="console" class="org.apache.log4j.ConsoleAppender"> 
    <param name="Target" value="System.out"/>
    <layout class="org.apache.log4j.PatternLayout"> 
      <param name="ConversionPattern" value="%-5p %c{1} - %m%n"/> 
    </layout>
  </appender>

  <root>
    <priority value ="debug" />
    <appender-ref ref="console" />
  </root>
  
</log4j:configuration>

We have to prepare appropriate filtering xslt file:

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

  <xsl:template match="root/priority/@value">
    <xsl:attribute name="value">warn</xsl:attribute>
  </xsl:template>

  <xsl:template match="@*|*">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="comment()">
     <xsl:copy />
  </xsl:template>

</xsl:stylesheet>

, and then call:

$ xsltproc -o log4j.xml filter.xsl log4j.xml 

The XSLT file could look a bit verbose, but in fact only first template match is specific. The rest will just copy xml from input to output. Using this technique we can also strip some attributes or semantically replace more structured XML fragments. And all of this without removing XML comments.

The only drawback is that DOCTYPE will not be preserved.

If you want to match specific attribute value:

  <xsl:template match="Connector/@port[.='8080']">
    <xsl:attribute name="port">8180</xsl:attribute>
  </xsl:template>

Nexus on debian

Nexus is very cool Maven Repository Manager, and debian is as cool server OS platform. I tried to match them. There was some pain, however finally I succeeded :).

Nexus is bundled as war now, thus the description is about installing it on tomcat.

In case of using debian etch (current stable), you may consider installing jdk 1.6 from backports. With jdk 1.6 you have to specify new JAVA_HOME=/usr/lib/jvm/java-6-sun in /etc/default/tomcat5.5.

Debian's tomcat has security manager turned on by default. Of course it could be easily disabled in /etc/default/tomcat5.5, however I feel safer when it is turned on :). Establishing security policy is not easy. My solution to the problem is based on work described in Mark Petrovic's article. I tweaked it a bit not to generate redundant rules. In case of nexus the resulting file looks as follows:

grant codeBase "file:${catalina.home}/bin/tomcat-juli.jar" {
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/WEB-INF/classes/logging.properties", "read";
};

grant codeBase "file:${catalina.base}/webapps/nexus/WEB-INF/lib/-" {
    permission java.util.PropertyPermission "*", "read,write";
    permission java.io.FilePermission "/", "read";
    permission java.io.FilePermission "${catalina.base}/logs", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.properties", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.xml", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/WEB-INF/log4j.properties", "read";
    permission java.io.FilePermission "${catalina.base}/temp", "read,write";
    permission java.io.FilePermission "${catalina.base}/temp/-", "read,write,delete";
    permission java.lang.RuntimePermission "defineClassInPackage.java.lang", "";
    permission java.lang.RuntimePermission "createClassLoader", "";
    permission java.lang.RuntimePermission "setContextClassLoader", "";
    permission java.lang.RuntimePermission "accessDeclaredMembers", "";
    permission java.lang.RuntimePermission "getenv.*", "";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.misc", "";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.reflect", "";
    permission java.lang.RuntimePermission "reflectionFactoryAccess", "";
    permission java.lang.RuntimePermission "getClassLoader", "";
    permission java.lang.RuntimePermission "modifyThread", "";
    permission java.io.FilePermission "${catalina.home}/sonatype-work", "read,write";
    permission java.io.FilePermission "${catalina.home}/sonatype-work/-", "read,write,delete";
    permission java.net.SocketPermission "*", "connect,resolve";
    permission java.lang.reflect.ReflectPermission "suppressAccessChecks", "";
    permission java.util.logging.LoggingPermission "control", "";
};

grant codeBase "file:${catalina.base}/webapps/nexus/WEB-INF/classes/-" {
    permission java.util.PropertyPermission "*", "read,write";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.properties", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/localhost/nexus/WEB-INF/plexus.xml", "read";
    permission java.io.FilePermission "${catalina.base}/webapps/nexus/WEB-INF/log4j.properties", "read";
    permission java.io.FilePermission "${catalina.home}/sonatype-work", "read,write";
    permission java.io.FilePermission "${catalina.home}/sonatype-work/-", "read,write,delete";
    permission java.lang.RuntimePermission "getenv.*", "";
    permission java.lang.RuntimePermission "defineClassInPackage.java.lang", "";
    permission java.lang.RuntimePermission "createClassLoader", "";
    permission java.lang.RuntimePermission "setContextClassLoader", "";
    permission java.lang.RuntimePermission "accessDeclaredMembers", "";
    permission java.lang.RuntimePermission "modifyThread", "";
    permission java.util.logging.LoggingPermission "control", "";
};

Copy these rules into 60nexus.policy file and place it in /etc/default/tomcat5.5. Restart of JVM is required to make these rules effective.

The line:

permission java.io.FilePermission "/", "read";

is redundant with other rules. The need of reading the whole filesystem seems to be a security flaw. I hope it will be eliminated in future nexus releases. I will fill the bug for this.

These security rules should work ok not only on debian, but also on any other system where tomcat is deployed. There are two system properties in use: catalina.home and catalina.base. This seems to be debian specific. In case of other systems catalina.home should be enough.

Nexus would create sonatype-work directory in user.home which is /usr/share/tomcat5.5 in case of debian. However with security manager we have to create it by ourselves.

host# mkdir /var/opt/sonatype-work
host# chown tomcat55:adm /var/opt/sonatype-work
host# ln -s /var/opt/sonatype-work /usr/share/tomcat5.5

The /var/opt/sonatype-work will become storage which should be backed up carefully when nexus is used not only as proxy, but also as local maven repository.

Now we can download the latest nexus from http://nexus.sonatype.org/using/download.html

Create logging.properties file with the following contents:

org.apache.juli.FileHandler.level = WARNING
java.util.logging.ConsoleHandler.level = WARNING

and put it into WEB-INF/classes inside the war.

Without this modification nexus would log every single HTTP request on console which would be redirected to catalina.out log file. To much verbose default logging seems to be another nexus bug.

Strip the version number from war file, stop tomcat with:

host# /etc/init.d/tomcat5.5 stop

Put nexus.war in /var/lib/tomcat5.5/webapps.

host# /etc/init.d/tomcat5.5 start

Nexus should be visible at http://debianhost:8180/nexus/

Upgradges should be as easy as putting new nexus war in webapps.