Every technology has its purpose (about XPath)

Recently I had to fix a bug in our production enviroment caused by a performance issue. Basically the problem was that we were missing a couple of requests by timeout because the process parsing a xml was taking too long, and the reason was the misuse of Xpath and the size of the xml (about 700 children nodes under the root element). Just to be clear, the xml is parsed and cached in lazy mode, and unless the file change we never parse it again.

When I started the tests in my old heavy loaded laptop, the process took 25 seconds in parse the xml, after the fix it took less than a second.

Our xml has the following structure:

<root>
<element id="1"/>
   <param id="1"/>
   <param id="2"/>
   ...
   <section>
      <sec-element id="1"/>
      <sec-element id="2"/>
      ...
      <sec-element id="n"/>
   </section>
</element>
...
</root>

We map the 'element' nodes as java objects so we iterate over each one of the elements and get the data, but to get the elements under the 'section' we use the following xpath expression "section/sec-element" on each 'element' node.

I guess I was lazy the day I coded the parser (yes, mea culpa) and I wanted to be sure the 'sec-element' nodes where under a 'section' node, so I decided to use xpath, and that was my mistake. The wikipedia says: "XPath, the XML Path Language, is a query language for selecting nodes from an XML document", but I wasn't using it for selecting nodes, I knew the nodes were there (and if I didn't, I should have used XML Schema), I wasn't searchin the nodes with an attribute or the ones with a specific attribute value, I just wanted all of them.

After remove the xpath querys for a plain getElementsByTagName, the problem was solved.

Just remember, before use a technology/tool make sure is the right one for the job, it will save you a lot of problems and time.

Standards matter

Yesterday our java snippet code to retrieve resources from a web server stopped working on our testing environment. The only change was the url configured, someone had decided stop using ip numbers in configuration files (a right decision in my opinion) so the url change from http://192.168.34.15:1234/path/file.txt to http://machine_8.internaldomain.com:1234/path/file.txt.

Of course the etc/hosts file had been properly configured so machine_8.internaldomain.com resolve as 192.168.34.15 but our code raised the following exception "java.net.ConnectException: Connection refused: connect". Reviewing the logs I found the code trying open a connection to machine_8.internaldomain.com:80 where obviously no server was listening.

The java snippet code use the commons-httpclient library (version 3.1), so my first thought was we have to update the library (version 4.1) but it didn't work. After restoring the original version of the library and spend a couple of hours debugging, I found the problem in the library URI object implementation (org.apache.commons.httpclient.URI) and I managed to code a workaround.

When I was about to share the workaround in this blog, I decided to test it with the last version of the library, but one of the changes from version 3.1 to 4.x was the use of the own jdk URI object implementation (java.net.URI). Since the java.net.URI object was introduced in jdk 1.4 and I was testing with java 6, I couldn't believe there was a real bug in the jdk.

After another couples of hours testing and debugging I found the real problem (extracted from wikipedia):

   "Hostname labels may contain only the ASCII letters 'a' through 'z' (in a case-insensitive manner), the digits '0' through '9', and the hyphen ('-')"

So the problem was us not following the standards and using an illegal character (the underscore '_') in the hostname. After changing everything from "machine_8.internaldomain.com" to "machine-8.internaldomain.com" everything started to work again.

Solve classpath format problems when launch java programs via shell scripts in cygwin

At work our java programs runs in unix machines, so we write shell scripts to launch them. Unfortunately our development environment is windows, so test the shell scripts is painfully because you have to install the entire application on the test server.

To avoid the installation of the application, i installed cygwin on my computer to easily test shell scripts, but there is a problem with the classpath format (explained here and here). As you can see in referenced posts cygwin provide an utility to solve this kind of problems (cygpath), so i came with a solution to execute my shell scripts without modify them.

First you have to write a shell script named java and placed it at the ~/bin directory:

#!/usr/bin/ksh

arguments=""
cpmod=0

for arg in $*
do

if [ $cpmod -eq 1 ] ; then
cp=`cygpath -wp ${arg}`
arguments="${arguments} ${cp}"
cpmod=0
else

if [[ $arg == "-cp" ]] || [[ $arg == "-classpath" ]] ; then
cpmod=1
fi
arguments="${arguments} ${arg}"

fi
done

java.exe ${arguments}

Finally you must set PATH variable enviroment to search first in your user bin directory. This can be easily done editing .bash_profile.