Integrating Spring MVC and a “Real” Database

On the Spring MVC and WebFlow video training course, we build a system from scratch based around a book store. To keep things focussed on the web tier, we use a mock implementation of a service class.

Therefore, we can run the system up without worrying about setting up a database, getting the tables set up etc.

However, I’ve had a few requests from people who are stuck on “upgrading” their Spring MVC project that uses the Mock. They want to talk to a back end database but for some reason it isn’t working.

The best way to work is to start from your Spring-MVC project, and make sure that is working. As on the course, you’ll probably have a Dispatcher-servlet.xml file with the web-tier beans, and a separate application.xml containing the “middle-tier” beans.

Now write a production version of your service class, in the same style as on the fundamentals course (for example, service class calls a dao which uses a Spring template to read/write to the db).

As all of the above is covered in both courses, it’s just a case of integrating your knowledge.

There are a couple of steps you might need to follow however:

1: Add in missing Jar files

Add in any JAR files that you need for your data access strategy. I used Hibernate 3 (the old XML mapping style, not annotations), and I added:


If you’re using Hibernate with Annotations, or MyBatis or whatever, your dependencies will be different. I simply borrowed these JARs from the project I created on the Spring Fundamentals course.

Step 2: Ensure your mapping files are copied across (Hibernate XML only)

Now for an annoying complication: I’m using the old fashioned but still usable XML mapping approach to Hibernate. I have a file called Book.hbm.xml in my package “com.virtualpairprogrammers.domain”.

However, this file is not being picked up by the build script we used on the Spring MVC course. When Eclipse compiles, it copies any non-Java files from the /src folder structure, across to the corresponding folder under /bin.

The ant compile task does NOT do this, however. So I need to manually add this step into my build.xml.

<!-- ensure hibernate XML files are also moved in the compilation -->
<copy todir="${bin}">
   <fileset dir="${src}">
      <include name="**/*.hbm.xml"/>
Note: this is added inside the compile target.

I’ve uploaded a full sample project to These sample files will only be available for a short period, so grab them while you can.

So What’s Wrong with Checked Exceptions?

In part one of this blog post on Checked Exceptions, I defended them. They have a very valuable place in Java’s toolbox, and those who argue for their removal are missing the point of them.

The trouble is, they’re overused. In the early days of Java, everyone assumed that Checked Exceptions were the “right” things to use in all circumstances.

But when you throw a checked exception, you are forcing the caller to handle the problem. But often, problems can’t be possibly be handled by program code. The number one example of this is when writing JDBC (data access) code:

   String sql = "INSERT BOOK (bookID, ISBN, title, author, price) VALUES (?,?,?,?,?)";

   PreparedStatement stmt = con.prepareStatement(sql);

   // bind the parameters - omitted
catch (SQLException e)

(This is just a fragment for illustration – don’t use this in any real code. I’ve omitted the closing of the connections – don’t even get me started on that nightmare).

Now, seriously, what code can you possibly put in this catch block? Firstly, there are hundreds of things that could go wrong here:

1) Network connection to database down (a squirrel has eaten a cable somewhere)
2) No access to database allowed (bad credentials)
3) SQL statement grammar incorrect – [in fact – it IS! I deliberately made a typo in the SQL]
4) Datatypes of data wrong
5) Our DBA has gone mad/got drunk and dropped the table for a joke
6) Constraint violation

I could go on. But the point is:

None of these problems can be handled in the catch block. How can your code possibly fix the SQL error in my hardcoded string?

So what CAN we do? Well, if we continue to use checked exceptions only, then we have to throw the exception to the caller. But think about it – how can our caller (whatever that is) deal with any of the six problems above? Almost certainly, they can’t so they will re-throw it too. And their caller will have to rethrow it, and so on until it reaches probably the top level of our call stack.

The worst aspect of this is that many Java programmers don’t get this – they think because they’ve got an exception, they MUST deal with it there and then (they forget it can be rethrown). Which is why you’ll often see this disgusting code in production:

   // SQL and JDBC as above
catch (SQLException e)

Which translates as “hide the problem in a log file somewhere and then let’s carry on as if all was well”.

Probably the best you can do is to manually convert it into an unchcked exception. Since JDK1.4, you can wrap exceptions so that you don’t lose the detail of the original cause:

   // SQL and JDBC as above
catch (SQLException e)
   throw new RuntimeException(e);
   // or you you create your own runtime exception like DataAccessException

SQLException should have been made an *Unchecked* Exception in the first place. This would have meant that we could let the problem rise up the stack until it reaches an appropriate level where we should handle it.

If you’re in the lucky position of writing web applications, then you need never catch the unchecked exception – because web applications run in a container like Tomcat, effectively the Tomcat code is the “top level” of your app – you simply configure Tomcat to catch all unhandled exceptions, and to report them to the user and the system admin.

This is why the Spring Framework supplies a wrapper layer around JDBC, which automatically catches the checked exceptions for you, and it throws them to your code as unchecked exceptions. We talk about this in detail on the Spring Framework Online Course

When to use Checked or Unchecked?

Checked and Unchecked Exceptions are two very different tools. In fact, I suspect that the confusion about them arises because they’ve been given very similar names (even worse, the words Checked and Unchecked don’t even appear in the API). The designers of Java should have given them clear distinction in the API.[1]

  • Checked exceptions are for situations where you know your method has reached a condition which means the end goal of the method can’t be achieved, but somewhere in the call chain we except the problem can be dealt with.
  • Unchecked exceptions are for situations where something has gone wrong, but due to the nature of the problem, it’s likely that the problem can’t be handled (or would be best caught with a global error handler). Perhaps a configuration problem (eg a vital config file is missing) or an environmental problem (eg a database table has disappeared), or maybe even a problem within the code (eg a hardcoded SQL statement is wrong).

The point is, they are different tools and both have their strengths and weaknesses. Java is unique in having Checked Exceptions, and it would be a shame to see them removed from a future version of Java (as some people want).

[1] The inheritance hierarchy is broken also. RuntimeException extends Exception, which technically means that RuntimeException is a special kind of Exception. When it really, really isn’t (perhaps you could argue that Exception is a special kind of RuntimeException). This means that whenever you catch raw “Exception”, you are also catching RuntimeExceptions, even those such as NullPointerException, which is definitely something you don’t want to do.

In Defence of Checked Exceptions

I’m often asked about exceptions in Java, and whether Unchecked Exceptions are “better” than Checked Exceptions.

This is probably because in our Spring Framework online course, I spend a while talking about Spring’s Data Access Exception strategy, where checked exceptions are converted to unchecked ones.

See also blogs like Bruce Eckel’s , which is fairly damning about Checked Exceptions.

As some of you may not be familiar with the difference between Checked and Unchecked exceptions, let’s look at the difference between them.

Checked Exceptions

A checked exception is one that extends the java.lang.Exception class, and the rule is:

“Any call to a method that might throw a checked exception MUST handle the exception”.

By “handle”, I mean it must either

a) Handle the problem in a try…catch block
b) Throw the exception to the next level up

This means, that unless you’re cheating by swallowing the exception somewhere (this means an empty catch block), or you’re throwing exceptions from main(), then checked exceptions are guaranteed to be handled by your code, somewhere in the call chain.

Unchecked Exceptions
These are exceptions that don’t need a try…catch block anywhere, although you can optionally try…catch them if you want to.

To define an unchecked exception, your exception definition will extend java.lang.RuntimeException.

The risk of using Unchecked Exceptions is that they might never get caught. If they don’t, your program crashes. I’ll talk about why unchecked exceptions are useful in the second part of this post.

Using Checked Exceptions

Surveying the Java blogging world (eg Bruce’s post above), it’s fair to say that Checked Exceptions have a terrible, terrible reputation and are hated by a large proportion of Java programmers. Whilst I don’t blame them, this is unfair. Checked Exceptions have a lot of value but they are overused.

On my live Java course, I use a simple example of a checked exception…

public class NoOrdersAvailableException extends Exception
   // nothing important to add

Which is thrown from a method something like the following:

public class OrdersQueue
   public Order getNextOrderInQueue() throws NoOrdersAvailableException
      // do a database query to find the order
      // omitted

      if (nextOrder == null) throw new NoOrdersAvailableException();
      return nextOrder;

Why does the getNextOrderInQueue() method need to throw an exception? Because it has reached a condition where there are no orders available, and therefore it cannot fulfil the task it said it was going to do. I think of the signature of the method as being a contract:

public Order getNextOrderInQueue()

Meaning, “if you call this method, with no parameters, then the next order in the queue will be returned”.

Notice that we cannot handle the problem in this method. There are no orders in the queue, and the only solution is for the user interface to report this back to the user. But this is not a user interface class, so the caller needs to handle the problem.

And there’s the key: the caller NEEDS to handle this problem, and if the caller somehow misses the problem, we would have a serious problem (possibly silent failure).

Here’s the caller:

public class UserInterface
   public void showNextOrder()
         Order requiredOrder = ordersQueue.getNextOrderInQueue();
         // now add the order to the user interface screen
      catch (NoOrdersAvaialableException e)
         // show an information message to the user

I admit the code is long winded, and typing in try..catch blocks is never fun. But it is at least readable, I can work out exactly what is happening from this code with little prior knowledge (this is why naming exceptions well is a crucial skill).

Without checked exceptions, we would have to handle nulls, or create blank Order objects, or have some kind of pre-check such as “isOrderReady”, which would be prone to race conditions (you get back true from isOrderReady, but by the time you’ve called getNextOrderInQueue(), some other process has stolen your order!).

So checked exceptions have a very valuable place in helping you to write readable, clean and safer code. But in my next blog post, I’ll describe where Checked Exceptions should NOT be used.

Spring MVC and WebFlow – Released!

Well, it took a while but I’m relieved that the course is now out. Full details available at the site

And you can preview the opening chapter over at YouTube

I’ll be uploading some more previews of the more interesting chapters later this week, and for those of you needing the DVD edition, it will be available on Amazon as soon as we get the stock to them.

One of the reasons for the delay was the sheer amount of information in Spring-MVC. Despite a long running time (8 hours), there are still one or two topics that I wish had made the final cut, so keep an eye out for a few blog posts over the next few weeks that will fill in some gaps.

Another reason was the last minute decision to add WebFlow – I’m very glad I did this because although I love WebFlow, there’s not that much out there to help beginners.

I hope you like the course!

Changes to the Spring-MVC Course – now with added WebFlow!

We’re getting closer to releasing the Spring-MVC course and as we reach the end it’s becoming clear that some juggling is needed to the back end of the course.

Many people have contacted me to ask about Spring WebFlow, and whether we’ll be doing a course on it. I’d always planned to do a separate standalone course on this topic, but on reflection I’ve decided it would fit more naturally as a chapter on the Spring-MVC course itself.

The revised running order is now:

1: Introduction
2: An overview of MVC
3: First steps in Spring MVC
4: Mapping Parameters, Accessing Sessions
5: Internationalization (i18n)
6: Form Handling and Validation
7: Validation and JSR-303
8: Alternative views and Ajax
9: Spring WebFlow – the Basics
10: Going further with WebFlow

The chapter on Post-Redirect-Get was too small to stand on it’s own and is now part of the form handling chapter.

I think this outline is a bit tighter and more focussed, and with WebFlow included, will be much better value.

It does, however, mean a slight delay to allow time for scripting and recording of the more advanced material, but not a big one, we’re now on for Friday 6 August.

Working with Ajax and JSON in Spring-MVC

I’m now working on the final major chapter of the Spring-MVC course, how to use Ajax with Spring.

I must say I’m pretty shocked (in a good way) at just how simple the support for Ajax in Spring-MVC is.

On the course, we’re going perform a search against a database of books, and produce a list of near matches on the fly, as the user types into a text box on the web form. Something like this, with the results updating as soon as I type a letter in the search box…

I’d like to be able to send the list of matching books back to the web browser in JSON form. Although in the early days of Ajax, data was sent in XML form (the X in Ajax), these days JSON is a more natural choice as it is more compact and much easier to manipulate on the JavaScript client.

For details of what JSON looks like, see here. For example, a collection of book data might look something like:

[{"author":"Will Self","title":"Great Apes"},
 {"author":"Charles Dickens","title":"Great Expectations "},
 {"author":"Ken Kesey","title":"Sometimes a Great Notion "},
 {"author":"F. Scott Fitzgerald","title":"The Great Gatsby "}]

So I’d like my controller to be able to send back the results in the form of JSON. You might be thinking that I need to convert my Java objects manually – but in fact Spring-MVC makes it almost trivial:

  1. Annotate your controller method with @ResponseBody. This annotation tells Spring that the response to the browser should be serialised (ie converted from plain Java Objects into some kind of flattened, encoded structure).
  2. Add the JAR files from the Jackson project into your lib directory. Jackson is capabale of transforming object graphs into JSON form.
  3. You also need the <mvc:annotation-driven/> directive in your Spring wiring. By the time we do Ajax on the course, we’ll have added this already, but if you’re reading the blog separately from the course, you’ll need to make sure you’ve done this.

These three things combined tells Spring that we want to send the response back from the method annotated with @ResponseBody in the form of a JSON graph.

So, the controller looks wonderfully simple:

public @ResponseBody List&lt;Book&gt; performLooseSearch(@RequestParam("CHARS")String chars)
    return bookService.searchBooksByLooseMatch(chars);

In terms of Spring-MVC, that’s all there is to know. You now just need to code a front end capable of making an Ajax call. Until recently, the recommended approach in Spring was to use a framework like DWR – a framework that allowed us to call Java code as if it were a JavaScript method.

These days, however, the amazing jQuery library is the preferred choice. jQuery is a library that makes everyday JavaScript tasks such as making Ajax calls almost trivially easy.

Now, jQuery is beyond the scope of Spring-MVC so I can’t provide a tutorial here, but I can show a code fragment that demonstrates how to call our loose search:

   &lt;script type="text/javascript" src="/scripts/jquery-1.4.2.min.js" /&gt;
   &lt;script type="text/javascript"&gt; 
  function doSearch() {   
     $.getJSON("", { CHARS: $('#searchBox').val() }, function(data) 
                  for (var index in data) {
                     $('#results').append('&lt;p&gt;' + data[index].title + '&lt;/p&gt;'); 

&lt;h1&gt;Loose Search Test&lt;/h1&gt;

&lt;input type="text" id="searchBox" onKeyUp="doSearch();"/&gt;

&lt;div id="results"&gt;
 Results will appear here...

All you need to do is download the jquery JavaScript file from here and make sure it is picked up by your page.

All references to $ are jQuery calls.

On line 7, getJSON performs an Ajax call to the URL “” – hence our controller will be run. The request parameter CHARS is populated with whatever is contained in the “searchBox” input field.

When (and if) a response is received by the server, the callback function that I’ve provided is invoked (lines 8-14). In the callback function, I simply clear the “results” div (line 9), and then I loop around the JSON results. For each element in the results, I extract the title and wrap it around a paragraph tag.

Line 21 ensures that the doSearch() function is called everytime I release a key in the input field.

If you’re interested in seeing this full example coded, the Spring-MVC video course is released on 30 July 2010. You can register an interest by sending a blank email to, and don’t worry, we won’t use your email address for any purpose except for keeping you updated with new courses.

The Spring IDE has a New Update Site

Edit July 2011: It seems the address is back up and running again, please post a comment on this post and I’ll monitor the situation. As of today, you can do the following:

  1. Help/Install New Sofware
  2. Add… button, then “” (without the quotes) for both the name and location
  3. Click “Core/Spring IDE”
  4. Click “Next”, “Next” and accept the licence, then finish.

On the Spring Fundamentals course, we look at the Spring IDE. The name is a bit misleading, it’s really just a plug in for Eclipse rather than a full blown IDE.

Nonetheless, it is still a powerful plug in. As well as validating your XML configuration (so you can trap wiring errors without running the application), you can also see a very well presented graph of your beans, making it very easy to spot wiring problems.

However, since we recorded the course the URL of the update site has changed ( now gives 404 errors).

The correct update site is now

(I should note that SpringSource, the company founded by the creators of the Spring Framework also supply a full IDE based around Spring and Eclipse called the SpringSource Tool Suite. We didn’t cover this on the course, so see for full details).

Java Web Application Security – Part Two, Form Based Authentication

In Part One of this series on Java Security, we made our “add-new-book.jsp” page accessible only to those logged in as administrators. At present, we’re using so-called “BASIC Authentication” which in essence is asking the browser to handle the login challenge.

This is acceptable in some applications, but in general it is rather weak. The login box presented by the browser is very basic. Let’s look at how it looks on Firefox first.

Basic but functional. Particularly cute is the challenge: “The site says ‘Please log in'”. Recall that ‘Please log in’ was the string we used in the <realm-name>. Yet Google Chrome presents our <realm-name> differently:

Ugly. BASIC Authentication is ok for quick and dirty apps, but we can do better. We’ll now upgrade to “Form Based Authentication”. To do this, we provide a JSP page with a username/password box.

Step 1: Write the Login Form

I won’t bother writing a pretty form – I’ll leave it to you to make it look gorgeous.

File: login.jsp

&lt;h1&gt;Please Login&lt;/h1&gt;

&lt;form method=&quot;post&quot; action=&quot;j_security_check&quot;&gt;
 &lt;p&gt;&lt;label&gt;Username:&lt;/label&gt; &lt;input type=&quot;text&quot; name=&quot;j_username&quot;/&gt;&lt;/p&gt;
 &lt;p&gt;&lt;label&gt;Password:&lt;/label&gt; &lt;input type=&quot;password&quot; name=&quot;j_password&quot;/&gt;&lt;/p&gt;
 &lt;input type=&quot;submit&quot; value=&quot;Login&quot;/&gt;

Note firstly the name of the action – j_security_check. I haven’t made this name up – it’s a part of the Java Web standard, and is a predefined servlet. It is responsible for performing the authentication check.

Also part of the standard are the two parameters that the security check servlet demands – j_username and j_password.

Now the form is built, we need one further JSP (or HTML) page. If the user fails to login correctly, we need to tell them they have failed, so we need a “failed login” page.

File: failed-login.jsp

&lt;h1&gt;Sorry, please try again&lt;/h1&gt;

Again, I’ll leave it to you to make this page look good!

Step 2: Configure

We now need to upgrade our web.xml to point to our form and failure page. As always, the XML is a bit verbose but routine…

File: web.xml


Don’t forget to remove your existing <login-config> with the BASIC <auth-method>.

Step 3: Test

The most important thing to remember is that we don’t need to navigate to the login page directly. As before, we try to navigate to the “add-new-book.jsp” page, and the server now knows that before access is allowed, the login.jsp page must be presented first.

So let’s deploy and see what happens – the screenshot shows what happens when we navigate to “add-new-book.jsp”.

Notice that the URL in the browser bar is the name of the target page, even though it is actually login.jsp that has been presented.

And after we type invalid credentials into the form…

So, thanks to the predefined j_security_check servlet, you can get a reasonable security system up and running in a Java Web Application without writing a single line of code. Remember that this is all part of the standard and will work on any server you choose to use.

At the moment, our usernames and passwords are hardcoded into a file on Tomcat. This is fine for development but we need better for production.

There are actually many different ways you can authenticate users – using single sign on, Facebook Authentication, OpenID and so on, but the most common approach at first would be to provide a database table of usernames and passwords.

I could talk in detail about how to do this, but it is specific to your application server and it is fairly well documented by most servers.

If you are using Tomcat, then the documentation for setting up your table is available here

Don’t be put off by the term “Realm”. A Realm is just a “strategy for authenticating users”.

Java Web Application Security – Part One, Basic Authentication

One topic that we didn’t cover in the Java Web Development course was how to secure your application.  In this post, I’ll show that every Java webserver comes with a basic security model that will address many project’s requirements.

I’ll assume that you’re already familiar with Java Web development, and are able to build and deploy to a server such as Apache Tomcat.

Let’s start with a regular webapp. Any Java web application will do. I’m using the application we build on the Java Web Development course.

At the moment, all users can access every page in the application. The aim is for the page “add-new-book.jsp” to be accessible to logged-in administrators only.

Step 1: Switch security “on”

The first step is to declare in the web.xml file that we want to use security in our application. We’re going to start with the simplest form of web security, BASIC authentication. You’ll be familiar with this even if you don’t realise it – this is where the browser pops up a simple username and password box.

   <realm-name>Please log in</realm-name>

“realm-name” is just a string that will appear in the login dialog box.

This config tells Tomcat to instruct your browser to pop up a username/password challenge if a secured resource is accessed.

Step 2: Declare Your Secure Resources

A secured resource is just a URL, and we declare a URL to be secure with the following addition to web.xml:

  <web-resource-name>Admin Pages</web-resource-name>

It’s rather verbose XML (as usual with web.xml), but it is fairly straightforward. The URL ending with “add-new-book.jsp” will require the user is logged into the role of “admin”.

Now we’ve added this protection, let’s deploy the app and try to add a new book.

As we haven’t set up a valid administrator, whatever we enter here will result in failure. This is reported as a HTTP error status 401, and by default we see the following page:

Step 3: Authentication

So we’ve locked down part of our app – but how can we open it up to the administrators?

Authentication is the mechanism a web site uses to determine who the user is and to which role they belong. unfortunately, authentication is not part of the Java web specification, so different servers implement authentication in different ways. I’ll describe how authentication is done in Tomcat – for other servers such as Resin or Jetty, you’ll need to check their reference manuals.

The easiest way to set up users and roles in tomcat is to edit the file in {tomcat_home}/conf/tomcat-users.xml. I’ve edited the contents of the file to look like this:

   &lt;role rolename="admin"/&gt;

   &lt;user username="rich"

So the server will recognise a user called “rich” as being a member of the “admin” role.

After making the edit, we then need to restart Tomcat. We can now visit the page add-new-book.jsp – as long as we enter the username and password above. Notice that it isn’t necessary to implement a separate login page or to have to go to a login page first – when we try to visit the protected resource, the process is handled automatically by Tomcat.

You may be horrified that we have hardcoded a user into a file on the server. Of course, in a real application we would want a more sophisticated authentication strategy such as a database lookup. In part three of the series I’ll show you how to do that on Tomcat. But changing the strategy is just a configuration change, so using this hardcoded file of usernames and passwords is perfect for your development environment. You can always switch to something more robust once you go live.

The next problem is that the login dialog box is not very professional looking. Basic authentication is just that – basic. In part two of this series, I’ll be showing you how to add Form Based Authentication to your web app.