Format XML string with Mac OS X

Switching from Linux and Windows to Mac OS X sometimes is not easy. And, if I can say, it's not even worthy, as Mac OS X is not so wonderful as many people say.
Anyway, I had to switch, and I'm learning how to do some things that I used to do before with my new Mac.

So, to keep it short, with Mac Os X there is a useful CLI utility to format an XML string. You can even format something that is in your clipboard and put it back in teh clipboard.

This is the command:

pbpaste | xmllint --format - | pbcopy

Bettinglife

With great pleasure I can announce the release of my new Grails application: Bettinglife.
It's a betting portal (in Italian), with a comparison of the odds of many bookmakers, (Italians and Europeans).


It's in beta, but the main functionality is already there.


It can be found clicking on www.bettinglife.it

Why I like TDD. And I don't use it.

The first time I really understood the meaning of Test First Development (TFD) was like some kind or revelation.
I was already using automated testing, but most of the times the tests were written after the code. The few times I tried to write tests first, well, the result was not so brilliant. A lot of wasted time for some bad testing code. But still I wanted to to understand all the hype around TFD.
So, I continued to try it in some form, and continued to read a lot of technical books and articles about testing.

And I still remember that I was on the train back form work, reading some technical book (I don't remeber exactly, but it was not a book about testing, maybe about Tapestry or Hibernate or some other framework), when I read something that helped me to understand it. Like often happens in those 'bang' moments, after you don't remember exactly what happened (and I don't remember exaclty what I've read), you just understand that you know something more. And I mean, you really 'know', fully understand something.

Anyway, back to the topic...
What I understood was that with test last you are testing what your code does: that the code does what you want it to do correctly. With test first, you are testing that the code that you are going to write does exactly what it's required to do. It's a small but significant difference.
With test last development, you are testing only that your code behaves like you want, but not like the specification wants. And since you already have written the code, sometimes you don't notice that what you code does is NOT exactly what it should do. You know, after you write something it's like a newborn child. And every baby is good to his mam.

But, again, I don't want to explain in details what TFD is. I'm sure that you will find someone on Google (or Bing if you prefer) better than me in explaining that.
I just wanted to say that in that moment I understood the benefits of writing test first. And for some time I tried to do it.
But then I realized that most of the time this wonderful technique cannot be used.

And the reason is simple.
WIth TFD you write tests first to test that your code does what it should do according to the specifications.
But most of the time you don't know it exactly before writing the code, as specifications are just some dim lights far away. The customer doesn't know exactly what he wants, and if he knows, he will surely change his mind soon after the first test.
It's always happened in the projects I've worked, even in the most structured one. And I've worked in a lot of projects in the last years, with teams ranging from 1 (only me) to 30 people.

You end up with some sort of 'waste time to understand specs and write test-waste time to code and understand that maybe you didn't understand the specs so well-change the test and the code-hope that who has to decide (maybe you) really decided this time-change it again-start form the beginning' cycle.

It's not bad, after all everyone says that you have to write and then refactor your code. Good, I like to refactor. But if you refactor just because you didn't know exactly what to do in the first place, and you have to refactor the tests too for that reason, then writing the tests is a waste of time.
You end up to some point whan you have not enough time to refacor the code AND the tests. So you start leaving the tests behind. Maybe you comment them, or just remove some assertions. Or simply delete some of them because the code under test is not there anymore. Anyway, at the end they are not perfectly in sync with the code they should test.

Am I saying that tests are just a waste of time ? Absolutely not. I write as much tests as I can. But not in the early stage of the development.

So, what is my current process these days ?
First I write the code, without any test, according to what was my understanding of the specs. Then, when I have to change it or rewrite it (and I always have to do it in early stages), i don't have to refactor some old and outdated tests. I don't write any test, unless I have to refactor some piece of code that seems quite stable. Are there any bugs ? Probably, but I don't care too much. When some functionality is quite stable and confirmed then I write the most complete test suite that is worth for that code. And not more.

So, just to end up, what I'm saying is that TFD, or even too extensive TLD, for my experience are not to be used in the first stages of a project, when the main architecture and specifications are not clear. You waste a lot of time in writing test code to test functionality that will not see the production stage.

I know this is not probably teh best way to go, but often we have limited resources, and like Bruce Springsteen said:

'Between our dreams and actions lies this world'

It's been a long time...

...since my last post.
Whew...more than one year. A busy year.
I know that you don't care about what I've done in the last year, and I won't tell you.

So, why am I back here on the blog ?
Well, when I decided to write a blog was just to keep some sort of technical notebook. Sometimes I had some particular problem for which I could not find a good solution on Internet, or simply I just wanted to take note on how to do something. Like in a schetchbook. But then, most of the time I simply forgot to write on the blog, even about some really interesting stuff.

So, the question again, why am I back here after all this time ?

In the last years I've worked on a lot of diferent projects, often with some really good people. Now, in the last few months, I've been mainly working from home, and I will for at least some more months.
And I miss all those coffee machine talks with those great developers, all those wonderful technology insight that came out in front of a drink or a beer at the Sports Bar. So, I'm writing here some of the things that I would say there. Not only a virtual notebook, but also a virtual coffee machine and a virtual Sports Bar. Almost the same as the real one.

Except for the beer, of course.

Groovy faster than Java ?

Just a few days ago I wrote a post saiyng that yes, maybe Groovy is not as fast as Java, but is most than suitable for a big class of applications. I also mentioned some attempts to increase Groovy performances, mainly Groovy++. Some days later a surprising article was published by Alex Tkachman, the lead developer of the Groovy++ project.

This article presents just the result of some rough performance tests run on Groovy++ and previously presented on the Groovy++ site. The test itself was a Groovy implementation of a simple file word counting algorithm implemented in many languages (among these Java, Scala, Clojure and Ruby).

The result was a big surprise: while the 'standard' Groovy version was about 5 times slower than the Java version, the Static Groovy version was (slightly) faster than Java. The concurrent version was of course even better.

So...what does this mean ?
Well..nothing.
This benchmark was just an homemade test, without any assumption to be meaningful or definitive. The Java cose was not optimized, unlike Groovy code, and the test was highly depending on I/O.
But what's worth to be noted is they are working on it, and it's starting to make sense to compare the performances of Groovy with the ones of some other mainstream languages.

Just a couple of months ago it seemed like Sci-Fi...

Groovy 1.7

Today Groovy 1.7 was released.

The main features are:

• Anonymous Inner Classes and Nested Classes (Java Style)
  
• Support for annotations on imports, packages and variables declarations
  
• Grape enhancements (now it's possible to add a @Grab annotation on inports or variable declarations and download the requested dependencies from a repository)
• Power asserts that provide a descriptive message of what was wrong, like in the Spock Framework (real cool !)
• AST viewer and AST Builder
• Customizable Groovy Truth through the implementation af the asBoolean() method
• Improved Groovy Console
• SQL batch updates and transactions

A more detailed description of the new features can be found in the Groovy site.

Is Groovy such terribly slow ?

A thing you hear often is that Groovy is not a good language because of its performances. Actually Groovy IS slow. Depending on the type of task, it can be from 10 - 15 times to 90 times slower than the equivalent Java version. And is usually much slower than Scala too. (By the way, Scala seems to be the hot language of the moment, what Groovy used to be 1 year ago...).
And the reason why Groovy is so slow is quite clear. To add all the metaprogramming magic, Groovy has to dispatch every method (even the ones that don't use metaprogramming) dynamically.
So, the best thing we can do is to stop using Groovy, go back to POF (Plain Old but Fast for those who love acronyms) Java and maybe start learning Scala...
Well...no.
There are a lot of reasons to keep using Groovy.
Performances are constantly improving at every new version. There is even an attempt to create a partially statically compiled version of Groovy (see the Groovy++ project). They are tryng to offer a better support to concurrency throught the introduction of the "Actors Pattern", present in languages like Scala and Erlang (see the GPars project). Maybe I will write a post on these two projects someday. So, there are a lot of efforts on this topic. And then there is a very important reason...the killer application of Groovy.
Grails.
Of course Grails is based on Groovy, and extensively uses Groovy metaprogramming capabilities: so Grails performances are not so exciting. But it will take benefit from the improving performances of Groovy. And they are also working on big improvements on the performance side. As announced in the recent Groovy and Grails Exchange, (by the way, great event!) they are working on the precompilation of the GSP, maybe one of the biggest bottleneck in Grails. The number of request per second that Grails will handle in version 1.2 will double compared with older versions. And then, is the language speed so important when your application works with databases ? And with the network ? The weakest parts are others, not the language itself. With the current version (1.1), without GSP optimization , Grails performances are already comparable with the ones of other popular web frameworks (Tapestry, Wicket, Seam). But Grails is far mor productive than most of other frameworks.

So, is Groovy a slow language ? Yes, it is. If I want to develop an performance critical application with tons of business logic and number crunching, Groovy is not my first choice. But you shall not use the same tool for every job. (and yes, the language is a tool, even if a very important one, to build applications). For other kind of applications, the boost of productivity will overcome the performance issues. Do you remember what C++ developers were saying when Java came up ?

Groovy Testing Article

This month the last article in the 'Groovy tools for Java developers' series was published on MokaByte. It was related to Groovy testing, and can be found here. This article is based on a talk I made at the JUG Milano some months ago, that can be found on SlideShare.

Modify XML with groovy

One quite common thing to do when working with XML is to manipulate nodes: remove or rename attributes, changing values, remove or append nodes, and so on.
With Groovy's XmlParser this is quite easy, as explained in the official documentation.
Here are some more complex examples.
Suppose you want to set an attribute value, or to add the attribute if it doesn't exist. You can use this code:

if (!node.attribute('myAtt') {
    node.attributes().put('myAtt', 'myValue')
} else {
    node.attributes()[myAtt] = 'myValue'
}

If you want to rename an attribute:

def origAttrs = node.attributes()
if (origAttrs.containsKey(oldAtt)) {
    origAttrs[myAtt] = origAttrs[oldAtt]
    origAttrs.remove(oldAtt)
}

Suppose you want to replace a child node with another one (in the same position). Since the children() method returns a list of nodes, you can do like this:

def newChild = new Node(node, 'newChild', ['att1': 'val1'], 'New Text')
def indexOfChild = node.children().findIndexOf {it.name() == 'oldChildName'}
if (indexOfChild != -1) {
    node.children()[indexOfChild] = newChild
    newChild.parent = node
}

In this case we replace the child if there is only one node with that name, otherwise we replace the first one. Of course it's quite easy to replace all the children with a given name.

You can add these methods to a category or to the metaclass of the Node object to have them ready whenever needed.

A regular expression to find XML tags

One common thing to do when dealing with XML files which are not well-formed, is to preprocess them to fix the problems. So you have to open them and to extract the xml tags. You can do this with a regular expression. The problem is that the regex has to find all XML tags but it should not match everything between < and >, because you could have the case when the text inside XML contains angular brackets. I couldn'd find on the web a regex that manages those cases, so I had to create one.
This regex covers most of these cases (except one...):

</?[A-Za-z][A-Za-z0-9]*(\\s+[a-zA-Z0-9]+=(\'|")?\\w*(\'|")?)*\\s*/?>

So, it matches every word starting and ending with angular brackets, that can have a / at the beginning or the end, starting with a letter followed by 0 or more letters or numbers. If there are attributes it look if there is at least one space, than a word, a = and another word. The double qoutes are left optional, since I want to catch also not-so-well-formed tags.

So for example, it matches <TAG>, </TAG>, <TAG/>, <TAG /> <TAG AT="OK">, <TAG AT=OK/> and so on.
It will not match anyway text like '< 10 mg and > 8 mg', '<100 and > 50' and so on.
The only problematic case is when you have a text like '<beta and=gamma > delta'. It's quite an unusual and weird case, so it isn't really a problem.

No such property: save - Grails bug quickfix

There is a serious GORM bug affecting Grails version 1.1.1. When you try to save a domain class of a plugin or embedded in another domai class, sometimes you get a

groovy.lang.MissingPropertyException: No such property: save for class: MyDomainClass.

This bug will be fixed in Grails 1.2, but there is a quick and nasty fix: you need to add the line

MyDomainClass.get(-1)

in the BootStrap.groovy.

Gradle Article

After the first article about Gant, I published a second article on Mokabyte. It's the first part of an introduction on Gradle.

Gant article on Mokabyte

It's been a long time since my last post, but I was quite busy doing a lot of thing...Anyway, I had the time to write an article on Gant. The article has been published on Mokabite , one of the most famous italian site about the Java world...it's in italian, but if you understand the language of Dante...

Using NetBeans Profiler with a Grails App

I had some problems finding out resources about how to use the NetBeans profiler with a Grails app.
NetBeans doesn't know how to attach the profiler to a Grails application, so you have to configure the Attach Mode: here is how to configure NetBeans 6.5.
- Deploy the war in the application server as usual (Tomcat for example)
- From the NetBeans menu choose: Profile > Attach Profiler...
- Click on the Attach Mode link
- Select J2EE Web app as Target Type and the application server type where the application is deployed
- Select Local as attach method and Direct as attach invocation
- Click next > next and choose auto integration
- Insert your application server install directory and your target jdk
- In the next page click the Perform button (NetBeans will change the startup script of the application server)
- Follow the instruction in the next page to start your application server (or let NetBeans start it for you)
- Click Finish and in the Profiler window click Attach

Now the profiler is attached to your application in your Application Server

Jug - Groovy Testing Techniques

Last week I made a second presentation about Groovy to the JUG Milano. It was not a continuation of the Grails tutorial I started last time ( which will be continued with a more 'hands on approach'), but was an introduction of the techniques that Groovy provides for testing Groovy and Java code. You can find the slide on Slideshare

Groovy Testing

View more presentations from Davide Rossi.

Lesson learned: HtmlUnit and https

While developing my free-time project with HtmlUnit I found a strange behaviour when connecting to an 'https' site. The problem was a strange cpu usage, nearly 100% after the end of the method (actually, it lasted until the end of the thread where the scraping was made...).
The problem was due to an incorrect release of Javascript resources after the use by HtmlUnit and was resolved simply adding these 2 statements at the end of the method, disabling Javascript support and closing all connections:


webClient.setJavaScriptEnabled(false)
webClient.closeAllWindows()


Well, this makes sense, and maybe they should be added at the end of every method where HtmlUnit is used...

Grails and Wired.com: another success story

Another success story about the adoption of Grails for big sites was announced by SpringSource: the 'Products Review' section of the Wired.com site was entirely developed using Grails. That's another win for Grails after his adoption for the site of the UK broadcaster Sky

Jug Grails tutorial

Last week I had the great pleasure to make a brief presentation about Grails to the JUG Milano, mainly about Domain Classes and GORM. You can find the slides on Slideshare

Grails: a quick tutorial (1)

View more presentations from Davide Rossi.

Java 5 advanced for loop and null values

A quick note on an argument that may be obvious, but I never run through until today.
The advanced for loop (the for(Type t : Collection) form), introduced in Java 5 doesn't check for null collections. What this means is that if you pass to your loop a null collection (possible, when you don't have total control on the arguments that are passed to your method), the for loop will throw a NullPointerException. This is a correct behavior, since usually you don't want to consider in the same way null collections and empty collections, but can be a source of errors (possibly runtime errors...) if you don't take care of null values by yourself.

Two hearts are better that one...

A technique we are using these day, along side with the Pomodoro techinque time boxing, is Pair Programming, even if we are using it in a very unstructured way. As with every agile programming technique, this one revealed its usefulness, but has some 'dark corners'. What is Pair Programming ? Essentially it's just two people working togheter on a single workstation, sharing a single keyboard: one programmer (the driver) write the code, the other (the observer) review the code, trying to find out potential problems, bugs, bad code or thinking about possible improvements in code or design.

Some researches and studies have proven that pair programming reduce bugs of about 50%, with about a 15% cost increase (which is good, as bug fixing is much more costly if done at a later time, maybe by a developer who does't know the code...). A nice side effect is just that code and design decisions become shared among the team (at least two persons...), reducing the management risk (no programmer is a bottleneck, being the only one to know critical parts of the system, anymore...). There are obviously other benefits, like training on the job for less experienced developers, less distractions, better design and coding due to double knowledge and experiences...

Along with all these benefits, there are some drawback (as usual...).
According to the tries we are doing we saw that not all activities are good for pair programming. For example, there are some jobs that would be made exactly in the same way by both developers (as a result of common experiences), where the design is well known and the code quite consolidate, or some repetitive tasks. In this case the bugs are naturally few, and are quite easy to resolve. So, the contribution of the observer is quite low. The risk is to bring some boredom to the pair, and the attention in the long term is reduced by the pair practice instead of increased.
Another important thing to remark, which is not always kept in the right consideration, is that not all developers like to work for the whole day in pair. Some developers like to work alone, at least for some time, when trying some new technology or when doing some common task.

Another thing to think about is how to apply pair programming along with the pomodoro technique. Working for 20 / 25 minutes side by side brings the natural question of how to organize the work: for example is good to make an advance design, shared by the two, before writing the code ? Or is better for a developer to start coding while explaining his point, and then to review the design together ? Or every developer must try to prove that his design is good before starting ? All these approaches can lead to good results or to bad results.

In the next days we will try to use some of these approaches and to find out which ones work better for our project, a first pace in the road to agility.