Minor bug in our Webservices course

I’ve discovered a minor fault in our Webservices course. We supply a JSON file containing a data graph – and there’s a missing curly bracket! This is important because without it, any attempt to record a call via the REST Shell will fail with a JSON properties exception.

The file should look like this:

{"call":
 {"notes":"Customer called to complain about late delivery.",
         "timeAndDate":"2014-12-05T04:00:00Z"},

 "actions":[{"details":"Return call.",
             "requiredBy":"2016-12-09",
             "owningUser":"rac",
             "complete":false},
            {"details":"Check handled ok",
             "requiredBy":"2016-12-25",
             "owningUser":"rac",
             "complete":false}
           ]
}

The missing curly bracket is added to the end of the line with the timeAndDate.

Many apologies for the error, I hope it hasn’t caused too many problems.

Decent settings for DBCP Connection Pools

The Spring Framework course from 2009 is the first course that we’ve re-recorded at VirtualPairProgrammers. Although surprisingly little has changed in Spring since then, we felt it was time to polish the course up a little, to use the latest Spring 4, and in particular to use a more modern format for the video – with the second edition you’ll be able to view it on iPads and mobile devices, as with most of our other courses.

Note: everyone who bought the first edition of the course will automatically receive the second edition on the day of release – currently slated for around the 14 March 2014, but there may be delays as we complete the editing process.


I have actually made very few changes from the original. One area that I felt worthy of update was in our choice of connection pool. In the first edition we used the Apache DBCP connection pool, largely because it was the pool of choice at that time for the reference manual.

Since then, it’s fair to say that DBCP has come in for a lot of criticism, and other pools such as C3PO, Proxool or the Tomcat pool have become more popular.

There’s a great debate about this at StackOverflow (see here: https://stackoverflow.com/questions/520585 – a shame they closed the question as “not constructive” because it most certainly was constructive).

In the end, however, I decided to continue using DBCP for the second edition, partly to keep consistency with the old course, but also because actually DBCP isn’t that bad – we’ve used it successfully on several large scale projects with high traffic.

I think the biggest problem with DBCP is that the defaults are so poor. If you configure DBCP with just a driver, url, user and pass, then you’re going to end up with a  pool that soon locks up.

On the re-recorded version I alert the viewers to this, and tell you that you really need to tweak the pool to bring it to a performant level. But there isn’t time on the course to get bogged down in this, so I pointed the viewers to this blog post, where some more sensible values can be found.

Our default settings are:

  • maxActive = 150
  • maxIdle = 10
  • minIdle = 5
  • initialSize = 5
  • minEvictableIdleTimeMillis = 1800000
  • timeBetweenEvictionRunsMillis = 1800000
  • maxWait = 10000
  • validationQuery = “SELECT 1”
  • testOnBorrow=true
  • testOnReturn=true
  • testWhileIdle=true

And you set each of these properties in the Spring XML in the same way you set the driver etc. Eg <property name=”maxActive” value=”150″/>

I’m not saying these values are good for any application – you need to test, tweak and tune, but at VPP we use these settings as a starting point, and they are in fact the exact settings we currently have on our live site. Our live site isn’t exactly high traffic in the Facebook/Google sense, but we do get heavy traffic when we release a new course, so these settings should be reasonably good for most average websites.

Having said that, you can also switch to other pools quite easily, but I wanted to capture these defaults somewhere.

New course released soon – Java Build Tools

Edit to add the course was indeed released on 26 September!

I’ve been working for the last few months on a course that many of our customers have asked for – a course that covers the two major Java build tools, Ant and Maven.

It will be available at VirtualPairProgrammers on 26 September 2013. I’ll be announcing it here and we’ll also be in touch if you’re on our mailing list, Facebook page and Twitter.

It has taken so long to record because a) I always take a long to time to record(!) but also b) both Ant and Maven contain so many little twists and turns, and I feel that any decent course should get at least a little bit deep.

I’m well aware that both Ant and Maven are a little bit old now (don’t get me wrong: they are both used in thousands of projects around the world – their value is enormous!), so I’ve spiced up the course with a third build tool – it’s much newer and much easier to use: Gradle.

Gradle isn’t used on as many projects, but I’m hoping it’s going to get more popular over time. Hopefully this course will help raise its profile a little!

The chapter list isn’t yet complete, but the structure will be:

  • Introduction: Why use build tools?
  • Part One: Ant. (around 3 hours across five chapters)
  • Part Two: Maven (again around 3 hours)
  • Part Three: Gradle (about 90 minutes and two chapters)

On all three parts of the course, I show how to create a build from scratch, and the end result is a web application deployed to Tomcat.

Once that’s released, I’m due to start a big new project…

Running our JavaEE course on Glassfish 4

We’ve had a few requests asking if our JavaEE course can be run on Glassfish 4 (at the time of writing, the latest version).

The quick answer is: yes, but be careful. You will be able to run all the way up to chapter 16 before you see any problems, and they are very minor. However, Glassfish 4 is not as good as version 3 at reporting errors, and you don’t gain any features that you will need to follow the course.

As the important thing is learning the fundamentals of JavaEE (and these haven’t changed in JavaEE 7), my advice is to install a Java 6 JDK, and then use the Glassfish 3.0 that we ship with the course. Glassfish 3.0 is more stable, and seems to run faster. You can always upgrade to a later version once you’ve finished the course and understand the concepts.

(Glassfish 3.0 uses JDK6, although I’ve blogged here about how you can use it with JDK7. You *can* use Glassfish 3.2 with JDK7, but there was a horrible bug in 3.2 that prevented redeployments – I blogged about that here).

However, some of you will want to use Glassfish 4 – perhaps you need an advanced feature, or your company/project are using. In that case, you can do the course just fine, but there are a few things to be aware of.

1: Spurious warnings and swallowed errors

You will notice as soon as you deploy an application that uses a database on the server (Chapter 10), you will get the following:

Command succeeded with Warning. Cannot create tables for application. The expected DDL file EmployeeManagement_employeeDb_createDDL.jdbc is not available. Cannot create tables for application EmployeeManagement. The expected DDL file EmployeeManagement_employeeDb_createDDL.jdbc is not available.

In fact, all this is saying is “we looked to see if you have a custom create tables script in a file, and you don’t”. But that’s not a problem, because we are using automatic creation of tables, and that will happen in the background. So don’t worry, it probably COULD create the tables.

So, it is a very annoying warning. But it gets worse. If you DO have an error in your application (for example, I forgot to annotate one of my injected EJBs), you will get exactly the same warning – but this time as an error. But it will give no further clue as to the real cause of the problem. For that, you need to check in the log (by default, this version logs directly to the console that you started the application server in). So don’t forget that your real problem is probably unrelated to creating tables. Check the console log, although you will have a lot of useless information to wade through.

For this reason, I advise avoiding using this version for the training – but if you decide to go ahead, budget for some extra debugging time.

2: Glassfish Libraries

On the course, I advise you to add external references to a large collection of jar files. This was due to a bug in the Glassfish 3, you now just need to add an external reference to gf-client.jar in the early chapters (you’ll need a few more later).

Note: gf-client.jar is now located in GLASSFISH_HOME/glassfish/lib

For the JSF chapters, you will need an external link to the GLASSFISH_HOME/glassfish/modules/javax.faces.jar

3: Differences in the UI

There are some minor changes to the UI. Thankfully the UI is hardly changed. But you may have to hunt around for a few menu items.

4: Chapter 16, SOAP Webservices

All chapters upto and including the SOAP webservice chapter should run as on the video. However, in Chapter 16 you must write a new class to represent your webservice. I blogged about this here – although you could get away with not doing so in earlier versions, you need to be careful to do this in Glassfish 4. It’s better engineering anyway!

5: Chapter 18, REST Webservices

In the video, we ask you to configure a servlet in your application. This servlet is provided by Glassfish, and it does the work of providing the web service to clients. This isn’t part of the JavaEE standard, and it is therefore subject to change. And it has. It was previously a Sun class, I guess for political reasons they’ve renamed it.

You will need to change your web.xml file. Substitute this for the corresponding XML that you add in this chapter:

<!-- Configuration for JAX-RS -->
  <servlet>
     <servlet-name>Jersey Web Application</servlet-name>
     <servlet->org.glassfish.jersey.servlet.ServletContainer</servlet-class>
   
     <init-param>
        <param-name>jersey.config.server.provider.packages</param-name>
        <param-value>com.virtualpairprogrammers.staffmanagement.rest</param-value>
     </init-param>

   <load-on-startup>1</load-on-startup>        
  </servlet>
  
  <servlet-mapping>
     <servlet-name>Jersey Web Application</servlet-name>
     <url-pattern>/webservice/*</url-pattern>
  </servlet-mapping>

Edit to add: our Blogging platform seems to add some odd characters in the extract above: if you see XML errors you can download my working web.xml from here.

6: Chapter 19, REST Client

In the training course, we use Jersey-Client version 1, this allows us to call REST based webservices from Java. Your Glassfish 4 is using Jersey version 2. You can continue to use exactly the same client, because the client doesn’t know or care that you have upgraded your server. (All the client is doing is issuing HTTP requests – there’s no problem with the version mismatch).

If you need to keep up to date on the client as well, you can download Jersey version 2 from here. If you do this, you will need the JAR files from their lib directory, plus all of the JARs in the ext directory.

Unfortunately, Jersey (client) version 2 uses a much altered API, and your client code will break. The changes aren’t that radical, but there are too many to list here (for example: all of the packages are renamed, and instead of calling web.get(), you have to call web.request().get()).

You can download the modified REST client here, and a new version of the class we wrote to test delete requests from here – compare these with the ones we used on the course.

However, to reiterate, you don’t have to use Jersey version 2 for the client. Version 1 will interoperate perfectly well with your new server.

Edit to add: I accidentally deleted the Client Application from our server, many apologies if you’ve tried to download this without success. It’s now been re-added.

7: Chapter 20, Security

These are the biggest changes because authentication isn’t specified in the JavaEE standard. Here are the changes:

* EJB Client: You need to change your external library to point to GLASSFISH_HOME/glassfish/modules/security-ee.jar (and change the reference to this file in your build.xml script). You will now import com.sun.enterprise.security.ee.auth.login.ProgrammaticLogin instead.

  When you add the user and password to the file security realm, be sure to use the “Server Config”. This didn’t exist on Glassfish 3.

  When you now call pl.login(username, password), you will find this method is now deprecated. You can still call it, but to avoid the deprecation warning, you can call pl.login(username, password.toCharArray());

* REST Client: You will now need to call:

Client client = ClientBuilder.newClient();
client.register(new HttpBasicAuthFilter("rac", "secret"));

Anything else?

A long post, but actually very few real changes. That’s as it should be – JavaEE usually introduces new features and avoids breaking changes. If I have missed anything, do contact me and I’ll update this blog post as and when things change!

Hibernate and JPA Training Course – new release

At last, after six months of hard work, our Hibernate and JPA Training Course is now available.


It took so long because I wanted to cover all of the major areas of Hibernate and JPA. We can’t cover every single bit of the API (I do encourage you to use the reference manual through the course), but I wanted to make sure that all of the problems that I’ve encountered with Hibernate is there on the course.

The best parts of the course are the three chapters towards the end where we look at how Hibernate integrates with real architectures. We look at basic web applications, Spring Applications and EJB Applications and it is surprising how easy it is to run into problems!

As always, code is provided and we’re here to support you if you run into problems on the course!

The outline is:

  1. Introduction
  2. Getting Started
  3. Persisting Objects
  4. Configuring Hibernate
  5. Manipulating Objects (Dirty Checking)
  6. More on Mapping
  7. Handling Crashes and Logging
  8. Relationships
  9. Collections
  10. Bi Directional Relations
  11. Many to Many (really a long worked practical)
  12. Equals and HashCode
  13. XML Mappings
  14. Java Persistence API
  15. Cascades
  16. Embedding Objects
  17. Queries – Part 1
  18. Queries – Part 2
  19. Queries – Part 3
  20. Criteria API – Part 1
  21. Criteria API – Part 2
  22. Inheritance
  23. Detaching and Merging
  24. Optimistic Locking and Versioning
  25. Pessimistic Locking
  26. Performance and Lazy Initialisation
  27. Tuning Perfomance
  28. First Level Cache
  29. Second Level Cache
  30. Web Applications
  31. Spring Applications
  32. EJB/Java EE Applications
  33. Course Review

Configuring CacheConcurrencyStrategy in JPA2

Edited to add – to clarify I’m using EhCache as my Cache Provider

This will be a slightly more technical post than usual, I’ve struggled through a few problems whilst writing a chapter on “Second Level Caching” for our new Hibernate and JPA course, and I wanted to capture the results of the struggles.

This will be a long post, so sorry for the waffle, but the executive summary is:

JPA2 has a @Cacheable annotation, but it allows no parameters to tune the CacheConcurrencyStrategy. It isn’t clear, but the default setting will be READ_WRITE.


This blog post assumes you already understand the basics of the Second Level Cache (2LC), so I’ll dive straight into the depths:

(Of course, if you don’t know about how 2LC works – then you’ll need our course! Get it from our website, the release date will be early February).

What is the CacheConcurrencyStrategy?

When configuring a 2LC using Hibernate, it is necessary to set the Concurrency Strategy for each cache region. This was either done in the XML mapping file, or using their @Cache annotation:

@org.hibernate.annotations.Cache(usage=CacheConcurrencyStrategy.READ_ONLY)
public class Tutor
{

   // etc
}

The Concurrency Strategy is quite an important decision, as it is a fine tune of the cache region which could affect performance. You can choose from:
  • READ_ONLY. This sets the cache region as a read only cache, and this means it can be very performant as the cache doesn’t have to worry about locking. If you try to modify any of these objects though, you will get an exception.
  • NONSTRICT_READ_WRITE. This allows you to modify cacheable objects, but the cache provider doesn’t need to implement a strict lock on the cache. This means there is a potential for stale objects (ie one transaction has modified an object, but another object picks up the old version of the object from the cache). Choose this for writeable objects, but only if you don’t care about stale data.
  • READ_WRITE. The “safest” but least performant option. The cache provider will lock the cache when the object is updated, ensuring that all transactions will see the most up to date version.
(There is also “Transactional” which only applies to distributed caches. I won’t even think about that right now).
Ok. So far so good, and we’ve always had to do this when working with Hibernate 2LCs.
However, for this new video course I want to use JPA2 annotations as much as possible, in order to keep with the standards. The odd thing is that replacement for org.hibernate.annotations.Cache is javax.persistence.Cacheable and it allows no parameters.

@Cacheable    // eeek - we can't specify anything here!
public class Tutor
{

   // etc
}

So, what will the concurrency strategy be if we can’t specify it?
Well, it took me a good few hours to work through it, but I eventually found a few Hibernate JIRA issues (see here and here).
The key is that Hibernate “asks” EhCache what it’s default strategy should be, and it does this via call to the  EHCacheRegionFactory.getDefaultAccessType() method. 
And at last we can get the answer – checking the JavaDoc for this method here, we can see the following:

Default access-type used when the configured using JPA 2.0 config. JPA 2.0 allows @Cacheable(true) to be attached to an entity without any access type or usage qualification.

We are conservative here in specifying AccessType.READ_WRITE so as to follow the mantra of “do no harm”.

So, if you just use @Cacheable you will get a READ_WRITE Cache Region. Safe, but possibly not as performant as you would like.

If you don’t like this, then until JPA supports this property (if it ever does – it is an implementation specific feature), then you will need to add the old org.hibernate annotation:

@Cacheable
@org.hibernate.annotations.Cache(usage=CacheConcurrencyStrategy.READ_ONLY)
public class Tutor
{


I guess you may as well remove the javax.persistence.Cachable annotation, it is really redundant.

I can’t decide if this is the fault of JPA2 or EhCache/Hibernate. I’m leaning towards the latter – why should we need to configure the Concurrency Strategy through the code when there’s a perfectly good place to do it, in the ehcache.xml file. This configures the details of each cache region, why isn’t it done in there?

Do feel free to comment and set me straight on anything I’ve missed or misunderstood!

Hibernate and JPA Course Progress

This course has now been released (on Feb 14 2013) – full details here

I’m very sorry for the long delay in blog posts – but I’ve been very busy recording the Hibernate course. I’m glad to report that despite taking a long time (I started in July!), the course is looking great, and getting close to completion (current estimate – end of January).

I’ve found the big problem with Hibernate is that whilst it’s easy enough to get up and running (you can persist your first object in no time), Hibernate is deep. But it’s deep in a different way to – say for example – Spring.

With Spring, you can learn the fundamentals and start working on a project. The depth in Spring comes from the fact that it’s huge and has many different features. But you don’t need to know all these features right away – if you have the fundamentals (mainly Dependency Injection), you can learn the pieces you need as you go along.

But with Hibernate, a little knowledge can be dangerous. Sure, you can persist a few objects, but you will soon hit a brick wall. And it’s very easy to blame Hibernate for being too complex/too buggy/too random – when it’s really a lack of knowledge causing your frustration.

I’ve been determined that in this course, I will cover everything you need to know to be fully competent in Hibernate, and so I’m not shying away from drilling really quite deep. 

A second problem is that it is easy enough to work with Hibernate when you’re opening a session in a simple method, doing a bit of persistence and then closing the session. But real architectures don’t do this, so I’ve recorded a chapter showing how Hibernate works in Web Applications, in JavaEE applications and in Spring Applications.

Yet another difficulty is that we really need to cover BOTH JPA and Hibernate, and that’s hard to do without confusing (the major textbook on Hibernate tries to cover both at the same time, and it’s a bit of a mess). So my compromise is that we start with classic Hibernate, and fully understand the Session API, and then halfway through the course we switch over to JPA and work exclusively with EntityManager. I hope this approach has worked.

Anyway, I’ll get back to it now and I will announce when it’s finished. I’ve still to do some quite hard stuff – Lazy Initialisation and Caching are two big ones that I’m particularly dreading, but the end is in sight. I hope the wait will be worth it!

Creating a SessionFactory in Hibernate 4

I’m working on the first major chapter of the Hibernate course for VirtualPairProgrammers and I’m working through the process of writing our first object to the database.

I’ve been using Hibernate for years, so I’m well used to the process of creating the SessionFactory, but in Hibernate 4 they’ve changed the process around – but the documentation on what to do is terrible. (Nb – this is at the time of writing, it may have improved by the time you read this).

Essentially, the buildSessionFactory() method on the Configuration class has been deprecated.

Instead, you have to pass in a “ServiceRegistry” into the buildSessionFactory method. The reasons for this aren’t clear to me (I can’t find any documentation explaining why, so if you know please tell me and I’ll give you a credit on the course!)

After a few attempts, my code kept crashing with the following warning:

WARN: HHH000181: No appropriate connection provider encountered, assuming application will be supplying connections

Which I think means “unable to get a connection”

The exception given was:

Exception in thread "main" org.hibernate.HibernateException: Connection cannot be null when 'hibernate.dialect' not set

This is a terrible error because it implies I haven’t set the Hibernate Dialect (the dialect is essentially the name of the database vendor). Actually, Hibernate will automatically work out the database dialect, but it needs a connection to the database to do this. So the error thrown is a symptom of the problem, rather than the problem itself.

Anyway, I got it working in the end, here’s the code I’ve used on the course:

private static SessionFactory sessionFactory;

public static SessionFactory getSessionFactory()
{
   if (sessionFactory == null)
   {
      Configuration configuration = new Configuration();

      ServiceRegistry serviceRegistry = new
         ServiceRegistryBuilder().applySettings
                  (configuration.configure().getProperties()).buildServiceRegistry();

      sessionFactory = configuration.buildSessionFactory(serviceRegistry);
   }
   return sessionFactory;
}

This will be in some kind of helper class.

I must admit that I’ve struggled to get this first chapter done as I’ve found it hard to pick my way through all the steps needed to get to your first running Hibernate application. Hopefully I’ll speed up now we’re up and running – progress will be reported on the blog.

My Next Course: Hibernate and JPA

This course has now been released (on Feb 14 2013) – full details here
I’ve been working on projects for the last few months, so it’s been a while since I created a new course.

I’ve agonised what to go for – there are so many vibrant and interesting topics I can cover, but I’ve decided to create a course covering both Hibernate and JPA in full detail.

We have already covered Hibernate in a little detail on our Spring Fundamentals course, and we do some JPA on the JavaEE course, but really we just provide overviews on those courses and we don’t get to a full professional standard.

Hibernate and JPA need a course in their own right, and it’s probably the most asked-for course from our customers, so I’m into the studio on Monday morning to start work.

As always, our video courses take a while to record and edit, we work hard to get them right, so it will be at least 2 months before we’re ready to release. In the meantime, here’s the draft running order – please contact me if there are additional topics that you want to see on the course!

  1. Getting Started
  2. Persisting objects
  3. Relationships in Hibernate/JPA
  4. Bidirectional Relationships
  5. Implementing equals() and hashCode() correctly
  6. Mapping Inheritance Hierarchies
  7. Embedding Objects
  8. Queries using HQL / JPQL
  9. Using the Criteria API
  10. Detached Objects and Merging
  11. Versioning Objects and Locking
  12. Understanding Lazy Initialization
  13. Hibernate/JPA in Web Applications (OpenSessionInView/OpenEntityManagerInView)
  14. Extended Sessions and EntityManagers
  15. Hibernate Caching (and the second level cache)
  16. Hibernate Validators
  17. Introducing Hibernate Search
  18. Auditing with Envers

Accessing Parameters from a Spring AOP 2 Pointcut

I recently had a question from one of our customers, asking how you can access and modify the parameters in Spring AOP 2 – we didn’t refer to this on the course as we concentrated on the most common uses of AOP – so here’s a quick blog post showing how to do this.

If you haven’t used our Spring Fundamentals training course, the full chapter covering AOP is available as a preview below (you can play this at full screen to see the full detail):


Modifying parameters in advice is a very powerful thing thing to do (be careful – powerful means dangerous!), but it can be achieved easily in Spring AOP 2, using the AspectJ syntax.

On the course, we write a performance timing advice class, using Around advice. I’ve taken the annotation based version from the course, and I’ve modified it to take one of the incoming parameters (the customerId) and to change it’s value before passing it on to the target method.

@Advice
public class ModifyIncomingParameterAdvice
{
 @Pointcut("execution(* com.virtualpairprogrammers.calls.CallHandlingService.recordCall(..)) && args (customerid, call, actions)")
 public void trapRecordCall(String customerid, Call call, Collection<action> actions) {}
 
 @Around("trapRecordCall(customerid, call, actions)")
 public Object changeTheIncomingCustomerId(ProceedingJoinPoint method, String customerid, Call call, Collection<action> actions) throws Throwable
 {
  System.out.println("Asked to record a call for customer id " + customerid);
    
  // run the target code - but with a different customer id!
  String newCustomerId = customerid + "NEW";
  
  // we can pass parameters as an object array - you must get the correct number of 
  // parameters as the target is expecting, or you get a runtime exception.
  Object returnValue = method.proceed(new Object[] {newCustomerId, call, actions});
    
  return returnValue;
 }
}


Notice the use of the args() in the pointcut. This is the important part of this technique. It is applying the pointcut only to invocations of methods called recordCall that have exactly three incoming parameters. And these three parameters are given names which we can use as parameters to the Around advice.

If you want to use the XML form of AOP (I prefer XML for AOP in Spring), then you would just change the pointcut in your Spring XML – a rough example follows:

 <!--  Advice -->
 <bean id="modifyIncomingParameterAdvice" class="com.virtualpairprogrammers.advice.ModifyIncomingParameterAdvice"/>
 
 <!--  AOP Configuration -->
 <aop:config>
  <aop:aspect ref="modifyIncomingParameterAdvice">
   <aop:around method="changeTheIncomingCustomerId"
             pointcut="execution(* com.virtualpairprogrammers.calls.CallHandlingService.recordCall(..)) &amp;amp;&amp;amp; args(customerid, call, actions)" />
  </aop:aspect>
 </aop:config>

Notice that I had to escape the ampersands (&) in the pointcut, as you will get an XML error otherwise.

Now, this is a made up and probably useless example, but I hope you can see the power of this. I guess that most projects will very rarely want to use this technique, but I’d be very interested if you’ve used this on your real world projects – do tell me if you have some examples.