How to attach a Debugger to a running Tomcat or Glassfish instance

This is a frequently asked question from many of our customers at Virtual Pair Programmers, so I thought a blog post to capture the details would be in order. I’ll focus on Tomcat and Glassfish here, because we use them on our courses – but the details are the same for other servers.

  • (for Glassfish) 1) Run the server under debug. The easiest way is to run as normal, then go to the admin console. Go to configurations -> server-config (not default config) -> JVM Settings. Click Debug Enabled. These options will be different on different glassfish versions (I’m on 4), but you should be able to find them. Check the port that the debugger will run on – it will be part of the debug options and usually the default is 9009

    (for Tomcat) 1) Add a JVM option called “agentlib” to your startup script. On our courses, we use a bootstrap script called startup.bat, and you can edit it to look like this:

    cd ./tomcat/bin/
    java -Dsun.lang.ClassLoader.allowArraySyntax=true -agentlib:jdwp=transport=dt_socket,address=9009,server=y,suspend=n -jar bootstrap.jar

    (note: we use a simple bat file for bootstrapping Tomcat on our courses to simplify support: if you’re not using this script, then you need to put the JVM options in a new file bin/setenv.bat. Full details can be found here: )

  • 2) Restart the server (in Glassfish, a link may appear at the top of the page you can click. Otherwise, run the stopserv script and then startserv)
  • 3) Remember to add breakpoints in your code where required *AND re-deploy*. I sometimes forget to do this and wonder why I don’t hit any breakpoints.
  • 4) Now you can attach Eclipse to the debugger:

    a) Debug icon -> Debug Configurations
    b) Click “Remote Java Application”
    c) click the tiny icon at the top left – it is for “new session”
    d) Enter the correct port number you noted earlier (we suggested 9009)
    e) click the debug button.

  • 5) I find this odd: you won’t see anything special at this stage, you have attached to the running server *in the background*. There won’t be a console window and you won’t switch to debug perspective.
  • 6) You now need to hit a breakpoint, so to do this, exercise your code. This may be visiting one of your webpages, or running a test harness.
  • 7) When your client code causes a break to trigger on the server code, your run should be interrupted with a request to switch to the debug perspective, and you can now step through the code as usual.

I hope that’s useful!

Writing a Custom HTTP Message Converter in Spring

The Spring Webservices course got so big that we had to cut a few minor topics, and I promised on the video that I would write some blog posts covering them. Here’s the first of them, how to write a “Custom Message Converter”.

You probably don’t need to do this very often – I’ve never had to do this “in real life”. But it is a useful exercise to get a better understanding of what those message converters are doing.

Recall that in Spring, a MessageConverter is a class that is capable of converting a regular Java domain object to a REST representation (and back again). Spring has a small set of default converters already built in, but the two main ones are for JSON (most common representation used in REST) and XML.

For this exercise, let’s assume that for some reason, our REST application needs to support YAML as well. YAML is Yet Another Markup Language (literally) that aims to be simpler than XML. It’s used a lot in Rails.

As a starting point, I’ve fired up the REST project that we built on the training course. I’ve also started up the standard Spring REST shell:

baseUri mywebapp
get /customers

< 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Thu, 12 Feb 2015 17:56:33 GMT
  "customers" : [ {
    "customerId" : "100029",
    "companyName" : "Acme",
    "email" : null,
    "telephone" : null,
    "notes" : "No Notes",
    "calls" : null,
    "version" : 1,
    "links" : [ {
      "rel" : "self",
      "href" : "http://localhost:8080/mywebapp/customers/customer/100029?fullDet

As on the course, if the client wants XML instead, they can change the accept headers:

headers set --name accept --value application/xml

And now we repeat the get request….

get /customers
> accept: application/xml

< 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Thu, 12 Feb 2015 18:04:06 GMT
><customerId>100029</customerId> ... lots of XML snipped

But there is no YAML message converter installed by default in Spring….

headers set --name accept --value application/yaml
get customers

> accept: application/yaml


So let’s write a YAML Message Converter!

Step 1: Add the JAR file for YAML

One Java YAML parser is called SnakeYAML ( You can download the JAR from there, but if you have done our course, I actually supplied this JAR in the “Additional JARs” folder. So pull it from there and add it to your build path.

This library is very easy to use. If you want to try it out, you can easily convert an object into YAML (and back again) in a test harness.

public class TestYaml 
 public static void main(String[] args)
  Customer c = new Customer("10012", "Acme","Notes");
  Yaml yaml = new Yaml();

This gives an output like this:

calls: []
companyName: Acme
customerId: '10012'
email: null
notes: Notes
telephone: null
version: 0

Step 2: Write the converter

This is the bulk of the work. To write a message converter, extend the Spring AbstractHttpMessageConverter, and override the three methods as below.

  • readInternal() describes how Spring should convert the data (YAML) into a Java object.
  • writeInternal() is the opposite – it generates a YAML String from a Java object (this will be done in a similar way to our test above).
  • The supports() method is used to determine whether the converter actually supports conversion to and from the type of object in question. You might decide that you’re not going to support collections for example. We’ll simply return true and support any object.

In the constructor, we call the superclass constructor, which requires a MediaType object to denote what the HTTP media type is. We’re supporting application/yaml.

The implementations of the read and write methods are fairly routine, we’re just using the SnakeYaml library. It takes a bit of fiddling with the API of the HttpInputMessage and HttpOutputMessage classes to get what you need. In the read method, the getBody() method returns a standard Java InputStream, which luckily SnakeYaml can accept. In the write() method, we have to convert the YAML String into a byte array so we can send it to the write() method of the HttpOuputMessage. It’s all a bit fiddly but straightforward in the end.

public class YamlMessageConverter<T> extends AbstractHttpMessageConverter<T>
 public YamlMessageConverter()
        super(new MediaType("application","yaml"));
 protected T readInternal(Class<? extends T> arg0, HttpInputMessage arg1)
   throws IOException, HttpMessageNotReadableException 
   Yaml yaml = new Yaml(new Constructor(arg0));
   T object = (T)yaml.load(arg1.getBody());
   return object;

 protected boolean supports(Class<?> arg0) {
  return true;
 protected void writeInternal(T arg0, HttpOutputMessage arg1)
   throws IOException, HttpMessageNotWritableException 
  Yaml yaml = new Yaml();
  String result = yaml.dump(arg0);  

Step 3: Register the converter

The magic that makes the default message converters automatically happen is the tag in your Spring configuration.

We can add our new YAML Converter into the this tag:

<!-- This will automatically switch on the default httpmessageconverters -->
 <mvc:annotation-driven content-negotiation-manager="contentNegotiationManager">
  <mvc:message-converters register-defaults="true">
   <bean class="com.virtualpairprogrammers.messageconverters.YamlMessageConverter"/>

Note: the “register-defaults=true” is needed – without it, the default converters will not be registered and you will end up with only the YAML one.

And that’s it. We can now deploy the application and test:

headers set --name accept --value application/yaml
get customer/100029

< 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/yaml
< Transfer-Encoding: chunked
< Date: Fri, 13 Feb 2015 12:55:47 GMT
calls: []
companyName: Acme
customerId: '100030'
email: null
notes: No Notes
telephone: null
version: 1

Our representation is now in YAML.

I hope this exercise may prove useful to someone – to be honest I’m not really interested in YAML, the main point of the exercise is to get an understanding of what those mysterious HttpMessageConverters are doing!

Minor bug in our Webservices course

I’ve discovered a minor fault in our Webservices course. We supply a JSON file containing a data graph – and there’s a missing curly bracket! This is important because without it, any attempt to record a call via the REST Shell will fail with a JSON properties exception.

The file should look like this:

 {"notes":"Customer called to complain about late delivery.",

 "actions":[{"details":"Return call.",
            {"details":"Check handled ok",

The missing curly bracket is added to the end of the line with the timeAndDate.

Many apologies for the error, I hope it hasn’t caused too many problems.

Decent settings for DBCP Connection Pools

The Spring Framework course from 2009 is the first course that we’ve re-recorded at VirtualPairProgrammers. Although surprisingly little has changed in Spring since then, we felt it was time to polish the course up a little, to use the latest Spring 4, and in particular to use a more modern format for the video – with the second edition you’ll be able to view it on iPads and mobile devices, as with most of our other courses.

Note: everyone who bought the first edition of the course will automatically receive the second edition on the day of release – currently slated for around the 14 March 2014, but there may be delays as we complete the editing process.

I have actually made very few changes from the original. One area that I felt worthy of update was in our choice of connection pool. In the first edition we used the Apache DBCP connection pool, largely because it was the pool of choice at that time for the reference manual.

Since then, it’s fair to say that DBCP has come in for a lot of criticism, and other pools such as C3PO, Proxool or the Tomcat pool have become more popular.

There’s a great debate about this at StackOverflow (see here: – a shame they closed the question as “not constructive” because it most certainly was constructive).

In the end, however, I decided to continue using DBCP for the second edition, partly to keep consistency with the old course, but also because actually DBCP isn’t that bad – we’ve used it successfully on several large scale projects with high traffic.

I think the biggest problem with DBCP is that the defaults are so poor. If you configure DBCP with just a driver, url, user and pass, then you’re going to end up with a  pool that soon locks up.

On the re-recorded version I alert the viewers to this, and tell you that you really need to tweak the pool to bring it to a performant level. But there isn’t time on the course to get bogged down in this, so I pointed the viewers to this blog post, where some more sensible values can be found.

Our default settings are:

  • maxActive = 150
  • maxIdle = 10
  • minIdle = 5
  • initialSize = 5
  • minEvictableIdleTimeMillis = 1800000
  • timeBetweenEvictionRunsMillis = 1800000
  • maxWait = 10000
  • validationQuery = “SELECT 1”
  • testOnBorrow=true
  • testOnReturn=true
  • testWhileIdle=true

And you set each of these properties in the Spring XML in the same way you set the driver etc. Eg <property name=”maxActive” value=”150″/>

I’m not saying these values are good for any application – you need to test, tweak and tune, but at VPP we use these settings as a starting point, and they are in fact the exact settings we currently have on our live site. Our live site isn’t exactly high traffic in the Facebook/Google sense, but we do get heavy traffic when we release a new course, so these settings should be reasonably good for most average websites.

Having said that, you can also switch to other pools quite easily, but I wanted to capture these defaults somewhere.

New course released soon – Java Build Tools

Edit to add the course was indeed released on 26 September!

I’ve been working for the last few months on a course that many of our customers have asked for – a course that covers the two major Java build tools, Ant and Maven.

It will be available at VirtualPairProgrammers on 26 September 2013. I’ll be announcing it here and we’ll also be in touch if you’re on our mailing list, Facebook page and Twitter.

It has taken so long to record because a) I always take a long to time to record(!) but also b) both Ant and Maven contain so many little twists and turns, and I feel that any decent course should get at least a little bit deep.

I’m well aware that both Ant and Maven are a little bit old now (don’t get me wrong: they are both used in thousands of projects around the world – their value is enormous!), so I’ve spiced up the course with a third build tool – it’s much newer and much easier to use: Gradle.

Gradle isn’t used on as many projects, but I’m hoping it’s going to get more popular over time. Hopefully this course will help raise its profile a little!

The chapter list isn’t yet complete, but the structure will be:

  • Introduction: Why use build tools?
  • Part One: Ant. (around 3 hours across five chapters)
  • Part Two: Maven (again around 3 hours)
  • Part Three: Gradle (about 90 minutes and two chapters)

On all three parts of the course, I show how to create a build from scratch, and the end result is a web application deployed to Tomcat.

Once that’s released, I’m due to start a big new project…

Running our JavaEE course on Glassfish 4

We’ve had a few requests asking if our JavaEE course can be run on Glassfish 4 (at the time of writing, the latest version).

The quick answer is: yes, but be careful. You will be able to run all the way up to chapter 16 before you see any problems, and they are very minor. However, Glassfish 4 is not as good as version 3 at reporting errors, and you don’t gain any features that you will need to follow the course.

As the important thing is learning the fundamentals of JavaEE (and these haven’t changed in JavaEE 7), my advice is to install a Java 6 JDK, and then use the Glassfish 3.0 that we ship with the course. Glassfish 3.0 is more stable, and seems to run faster. You can always upgrade to a later version once you’ve finished the course and understand the concepts.

(Glassfish 3.0 uses JDK6, although I’ve blogged here about how you can use it with JDK7. You *can* use Glassfish 3.2 with JDK7, but there was a horrible bug in 3.2 that prevented redeployments – I blogged about that here).

However, some of you will want to use Glassfish 4 – perhaps you need an advanced feature, or your company/project are using. In that case, you can do the course just fine, but there are a few things to be aware of.

1: Spurious warnings and swallowed errors

You will notice as soon as you deploy an application that uses a database on the server (Chapter 10), you will get the following:

Command succeeded with Warning. Cannot create tables for application. The expected DDL file EmployeeManagement_employeeDb_createDDL.jdbc is not available. Cannot create tables for application EmployeeManagement. The expected DDL file EmployeeManagement_employeeDb_createDDL.jdbc is not available.

In fact, all this is saying is “we looked to see if you have a custom create tables script in a file, and you don’t”. But that’s not a problem, because we are using automatic creation of tables, and that will happen in the background. So don’t worry, it probably COULD create the tables.

So, it is a very annoying warning. But it gets worse. If you DO have an error in your application (for example, I forgot to annotate one of my injected EJBs), you will get exactly the same warning – but this time as an error. But it will give no further clue as to the real cause of the problem. For that, you need to check in the log (by default, this version logs directly to the console that you started the application server in). So don’t forget that your real problem is probably unrelated to creating tables. Check the console log, although you will have a lot of useless information to wade through.

For this reason, I advise avoiding using this version for the training – but if you decide to go ahead, budget for some extra debugging time.

2: Glassfish Libraries

On the course, I advise you to add external references to a large collection of jar files. This was due to a bug in the Glassfish 3, you now just need to add an external reference to gf-client.jar in the early chapters (you’ll need a few more later).

Note: gf-client.jar is now located in GLASSFISH_HOME/glassfish/lib

For the JSF chapters, you will need an external link to the GLASSFISH_HOME/glassfish/modules/javax.faces.jar

3: Differences in the UI

There are some minor changes to the UI. Thankfully the UI is hardly changed. But you may have to hunt around for a few menu items.

4: Chapter 16, SOAP Webservices

All chapters upto and including the SOAP webservice chapter should run as on the video. However, in Chapter 16 you must write a new class to represent your webservice. I blogged about this here – although you could get away with not doing so in earlier versions, you need to be careful to do this in Glassfish 4. It’s better engineering anyway!

5: Chapter 18, REST Webservices

In the video, we ask you to configure a servlet in your application. This servlet is provided by Glassfish, and it does the work of providing the web service to clients. This isn’t part of the JavaEE standard, and it is therefore subject to change. And it has. It was previously a Sun class, I guess for political reasons they’ve renamed it.

You will need to change your web.xml file. Substitute this for the corresponding XML that you add in this chapter:

     Jersey Web Application

     Jersey Web Application

Edit to add: our Blogging platform seems to add some odd characters in the extract above: if you see XML errors you can download my working web.xml from here.

6: Chapter 19, REST Client

In the training course, we use Jersey-Client version 1, this allows us to call REST based webservices from Java. Your Glassfish 4 is using Jersey version 2. You can continue to use exactly the same client, because the client doesn’t know or care that you have upgraded your server. (All the client is doing is issuing HTTP requests – there’s no problem with the version mismatch).

If you need to keep up to date on the client as well, you can download Jersey version 2 from here. If you do this, you will need the JAR files from their lib directory, plus all of the JARs in the ext directory.

Unfortunately, Jersey (client) version 2 uses a much altered API, and your client code will break. The changes aren’t that radical, but there are too many to list here (for example: all of the packages are renamed, and instead of calling web.get(), you have to call web.request().get()).

You can download the modified REST client here, and a new version of the class we wrote to test delete requests from here – compare these with the ones we used on the course.

However, to reiterate, you don’t have to use Jersey version 2 for the client. Version 1 will interoperate perfectly well with your new server.

Edit to add: I accidentally deleted the Client Application from our server, many apologies if you’ve tried to download this without success. It’s now been re-added.

7: Chapter 20, Security

These are the biggest changes because authentication isn’t specified in the JavaEE standard. Here are the changes:

* EJB Client: You need to change your external library to point to GLASSFISH_HOME/glassfish/modules/security-ee.jar (and change the reference to this file in your build.xml script). You will now import instead.

  When you add the user and password to the file security realm, be sure to use the “Server Config”. This didn’t exist on Glassfish 3.

  When you now call pl.login(username, password), you will find this method is now deprecated. You can still call it, but to avoid the deprecation warning, you can call pl.login(username, password.toCharArray());

* REST Client: You will now need to call:

Client client = ClientBuilder.newClient();
client.register(new HttpBasicAuthFilter("rac", "secret"));

Anything else?

A long post, but actually very few real changes. That’s as it should be – JavaEE usually introduces new features and avoids breaking changes. If I have missed anything, do contact me and I’ll update this blog post as and when things change!

Hibernate and JPA Training Course – new release

At last, after six months of hard work, our Hibernate and JPA Training Course is now available.

It took so long because I wanted to cover all of the major areas of Hibernate and JPA. We can’t cover every single bit of the API (I do encourage you to use the reference manual through the course), but I wanted to make sure that all of the problems that I’ve encountered with Hibernate is there on the course.

The best parts of the course are the three chapters towards the end where we look at how Hibernate integrates with real architectures. We look at basic web applications, Spring Applications and EJB Applications and it is surprising how easy it is to run into problems!

As always, code is provided and we’re here to support you if you run into problems on the course!

The outline is:

  1. Introduction
  2. Getting Started
  3. Persisting Objects
  4. Configuring Hibernate
  5. Manipulating Objects (Dirty Checking)
  6. More on Mapping
  7. Handling Crashes and Logging
  8. Relationships
  9. Collections
  10. Bi Directional Relations
  11. Many to Many (really a long worked practical)
  12. Equals and HashCode
  13. XML Mappings
  14. Java Persistence API
  15. Cascades
  16. Embedding Objects
  17. Queries – Part 1
  18. Queries – Part 2
  19. Queries – Part 3
  20. Criteria API – Part 1
  21. Criteria API – Part 2
  22. Inheritance
  23. Detaching and Merging
  24. Optimistic Locking and Versioning
  25. Pessimistic Locking
  26. Performance and Lazy Initialisation
  27. Tuning Perfomance
  28. First Level Cache
  29. Second Level Cache
  30. Web Applications
  31. Spring Applications
  32. EJB/Java EE Applications
  33. Course Review

Configuring CacheConcurrencyStrategy in JPA2

Edited to add – to clarify I’m using EhCache as my Cache Provider

This will be a slightly more technical post than usual, I’ve struggled through a few problems whilst writing a chapter on “Second Level Caching” for our new Hibernate and JPA course, and I wanted to capture the results of the struggles.

This will be a long post, so sorry for the waffle, but the executive summary is:

JPA2 has a @Cacheable annotation, but it allows no parameters to tune the CacheConcurrencyStrategy. It isn’t clear, but the default setting will be READ_WRITE.

This blog post assumes you already understand the basics of the Second Level Cache (2LC), so I’ll dive straight into the depths:

(Of course, if you don’t know about how 2LC works – then you’ll need our course! Get it from our website, the release date will be early February).

What is the CacheConcurrencyStrategy?

When configuring a 2LC using Hibernate, it is necessary to set the Concurrency Strategy for each cache region. This was either done in the XML mapping file, or using their @Cache annotation:

public class Tutor

   // etc
The Concurrency Strategy is quite an important decision, as it is a fine tune of the cache region which could affect performance. You can choose from:
  • READ_ONLY. This sets the cache region as a read only cache, and this means it can be very performant as the cache doesn’t have to worry about locking. If you try to modify any of these objects though, you will get an exception.
  • NONSTRICT_READ_WRITE. This allows you to modify cacheable objects, but the cache provider doesn’t need to implement a strict lock on the cache. This means there is a potential for stale objects (ie one transaction has modified an object, but another object picks up the old version of the object from the cache). Choose this for writeable objects, but only if you don’t care about stale data.
  • READ_WRITE. The “safest” but least performant option. The cache provider will lock the cache when the object is updated, ensuring that all transactions will see the most up to date version.
(There is also “Transactional” which only applies to distributed caches. I won’t even think about that right now).
Ok. So far so good, and we’ve always had to do this when working with Hibernate 2LCs.
However, for this new video course I want to use JPA2 annotations as much as possible, in order to keep with the standards. The odd thing is that replacement for org.hibernate.annotations.Cache is javax.persistence.Cacheable and it allows no parameters.
@Cacheable    // eeek - we can't specify anything here!
public class Tutor

   // etc
So, what will the concurrency strategy be if we can’t specify it?
Well, it took me a good few hours to work through it, but I eventually found a few Hibernate JIRA issues (see here and here).
The key is that Hibernate “asks” EhCache what it’s default strategy should be, and it does this via call to the  EHCacheRegionFactory.getDefaultAccessType() method. 
And at last we can get the answer – checking the JavaDoc for this method here, we can see the following:

Default access-type used when the configured using JPA 2.0 config. JPA 2.0 allows @Cacheable(true) to be attached to an entity without any access type or usage qualification.

We are conservative here in specifying AccessType.READ_WRITE so as to follow the mantra of “do no harm”.

So, if you just use @Cacheable you will get a READ_WRITE Cache Region. Safe, but possibly not as performant as you would like.

If you don’t like this, then until JPA supports this property (if it ever does – it is an implementation specific feature), then you will need to add the old org.hibernate annotation:

public class Tutor

I guess you may as well remove the javax.persistence.Cachable annotation, it is really redundant.

I can’t decide if this is the fault of JPA2 or EhCache/Hibernate. I’m leaning towards the latter – why should we need to configure the Concurrency Strategy through the code when there’s a perfectly good place to do it, in the ehcache.xml file. This configures the details of each cache region, why isn’t it done in there?

Do feel free to comment and set me straight on anything I’ve missed or misunderstood!