New Course: Kubernetes Microservices

Second course to be published at Udemy: Kubernetes Microservices.

I’ve set up a coupon code for a great deal.

Unlike the previous Docker course, this one is independent of programming language (Angular 6 used on the front end)

– although of course I can’t help dropping in a few references to Java along the way.

It’s a 12 hour hands on project: scenario – developers have pushed a set of images for a microservice system. Your mission, should you choose to accept it: define a Kubenetes workload locally, then deploy it to the cloud. Build in a level of resilience and then set up a proper ElasticSearch basic logging and monitoring system.

At the end of the course you’ll use your monitoring system to discover the developers did a pretty bad job in one of their images!

It was fun to record, hope it will be fun to watch and more importantly, a great set of real world skills for you.

A version of the course will appear in around a week at virtualpairprogrammers too.


Docker: Hands On Available at Udemy

I’ve just released my first course on Udemy. It’s a combination of the two modules I’ve previously published at VirtualPairProgrammers.

If you’re already a subscriber at VirtualPairProgrammers don’t buy the course again – but if not this course is a great way to get some real hands-on experience of Docker in a production cloud environment. I’ve set up a discount code for Udemy, follow this link for a great deal.

Kubernetes next up in a week or so!

Docker Build Hanging?

TLDR; create a .dockerignore file to filter out directories which won’t form part of the built image.

Long version: working on the upcoming Kubernetes course, with a massive deadline looming over me [available at VirtualPairProgrammers by the end of May, Udemy soon afterwards], the last thing I need is a simple docker image build freezing up, apparently indefinitely!

A quick inspection of running processes with Procmon (I’m developing on Windows) showed a massive number of files being read and closed:

This just a small extract from the process log.

This [every file in the repository being visited] is normal behaviour of the docker image build process – I guess I’ve just been working on repositories with a relatively small number of files (Java projects). This being an Angular project, one of the folders is “/node_modules” which contains a masshoohive number of package modules – most of which aren’t actually used by the project but are there as local copies. This directory can be easily regenerated and isn’t considered part of the project’s actual source code. [edit to add, it’s the equivalent of a maven repository in Javaland. The .m2 directory is stored outside your project, so this isn’t a concern there].

Turns out, the /node-modules folder contained 33,335 files whilst the rest of the project contained just 64 files!

Routinely, we .gitingore the /node_modules folder, and of course it makes sense to ignore this directory for the purposes of Docker also.

Simply create a folder in the root of the project, .dockerignore:

$ cat .dockerignore

You might also consider adding .git to this ignore list.

Now my docker image build is taking a few seconds instead of several hours. Perhaps I might meet this deadline after all…

Problems with dockerHost in Maven?

A problem reported against my Docker Module 2 course:

This happens on Windows Toolbox (ie anything below Windows 10 Pro) and possibly Mac toolbox as well.

If your Maven build is failing (with the fabric8 plugin) you need to add additional configuration. On the Docker Toolbox command line type

docker-machine env

Observe the output: you’re looking for the entries for DOCKER_HOST and DOCKER_CERT_PATH. Add the values you find here into your Maven pom, something like this:

...images tag etc here

Of course don’t just copy and paste these values – check they match the output of docker-env.

AllThingsJava Podcast

I’m back on the AllThingsJava podcast again. I had a shocking hangover when recording it so forgive my vague rambling.

Covered on this episode:

  • News – Javascript App gets sued for trademark breach – could it happen to us?
  • News – Java 10 features
  • News – (actually old news but since the last podcast) JavaEE is dead, long live JakartaEE.
  • Matt is planning a new course on Kotlin, coming soon.
  • I’m planning a new course on Kubernetes, coming even sooner. I’ll blog about this new course shortly. I should have done Kubernetes a year ago, I feel somewhat left behind and there are other good courses out there – I’m not sure how I can be different so I’m working on an “angle” right now. Hope I come up with something worth having.
  • There was some talk about whether we should do more on Javascript front ends. I think we should and I’m working on shoring up my skills in this area.

New Podcast – Java vs Scala

A new podcast is now available over at AllThingsJava. I’m not on it, which makes it even better than usual. The guest is Jon Humble, an experienced Scala developer.

We first met him when he presented a very funny talk which pitted Java against Scala in a kind of battle. Neither won because the presenters were too nice to declare a winner, so it was left to the viewer to decide…

…unlike me, Jon knows what he’s talking about so check out the podcast, lots of Kafka and similar buzzword goodness in there.

Java 10 – Local Variable Type Inference

In a previous post I listed the features added to Java 10. Naturally in a six-month update, the list isn’t long and most of them are updates to the runtime (such as very welcome garbage collector updates or internal changes).

But the number 1 headline grabber is the addition of type inference for local variables, and it’s a biggie. Let’s take it for spin.

What is type inference?

You’re all familiar with the following ritual:

 Customer customer = new Customer("Alan");

Which, if you think about it, is a bit absurd. We have to tell the compiler twice what type of object we’re creating. There is a reasoning behind this – technically the left hand side is telling the compiler what type we want the reference to be stored as, whilst the right hand side is the concrete object we’re creating. Sometimes, the difference is important and relevant – when programming to interfaces.


AbstractCustomer customer = new CreditCustomer("Alan");

Here want to instantiate a specific type of customer, but for the rest of the code, we only want to call the methods that are common to all customers. Polymorphism, blah blah blah. We’re not interested in that today, I just wanted to point out there is a reason for the left-hand-side-right-hand-side apparent duplication.

Here’s the thing – for the most part the two sides of the equation are duplicated. I want a customer and it needs to be of type customer.

Don’t Repeat Yourself is a bad thing and it’s almost like traditional Java wants us to repeat ourselves!

Anyway, stop waffling and cut to the chase. From Java 10 onwards, we can now let the compiler guess – or infer the type that we want…

 var customer = new Customer("Alan");

“var” is one of those rare things in Java, a brand new keyword! Java will now guess (it’s not hard is it?) that the type we want is Customer.

That’s it.

For the rest of the post I’ll describe a more realistic use of var. I’m picking an example from Spark, but don’t worry if you haven’t used Spark before, the example is actually generic.

A Real World Example

As usual with these toy examples, things don’t seem very groundbreaking but I believe this feature will have a major and positive impact on code productivity. Consider this monster from my Spark Training Course:

result = totals.mapToPair(tuple -> new Tuple2<Long, String> (tuple._2, tuple._1 ));

Now, I should know what the return type from this method call is. Erm, well, I should but it’s a bit of a headache. I know that the return class is some kind of “RDD”, I think a JavaRDD. And I can guess from the <Long, String> that the resulting return type will also have the same generic. So, I nervously type into the IDE:

JavaRDD<Long, String> result = totals.mapToPair(tuple -> new Tuple2<Long, String> (tuple._2, tuple._1 ));

Ouch. No! I now have a compile error…the type on the Left Hand Side is wrong. So now we reach a ludicrous situation: the compiler knows what the type should be, and I don’t. And the compiler won’t let me proceed until I guess correctly. (Ok, what I mean is – until I work out the right type).

Often on the Spark course I advise the viewer to simply leave off the return type and then use Quick Fix for the IDE to add in what it thinks is the right type!

Using the “Create Local Variable” Quick Fix should add in the left-hand-side declaration automatically.

I only advise using Quick Fixes if you know what you’re doing – in this case I’m basically asking Eclipse to do a type inference for me. But it’s very hit and miss – in this case, it gets it wrong. Or sort of half right:

For some reason (I care not why) the generics on the left hand side haven’t been inferred so the compiler is still moaning. A second “quick” fix will rectify this…

And – hooray, we finally have the right answer – it seems I had forgotten that the type was actually “JavaPairRDD”:

 JavaPairRDD<Long, String> result = totals.mapToPair(tuple -> new Tuple2<Long, String> (tuple._2, tuple._1 ));

But what a performance! And obviously the compiler knew the right answer all along. It was just teasing us.

So, to the point – when working in real coding situations like this, it can be easy to be unsure of the correct type you need for the declaration – and actually knowing the right type isn’t particularly illuminating. So, in Java 10 I could have avoided all that mess and jumped straight to:

 var result = totals.mapToPair(tuple -> new Tuple2<Long, String> (tuple._2, tuple._1 ));

Much cleaner and I hope in the long run, simpler. Note that none of this destroys strong typing – the type of result is still the same as it was, and we can only call the methods defined in the JavaPairRDD class, exactly as before.

I hope these kind of modern features are going to come thick and fast to future Java versions and that it will prove a compelling reason to upgrade – only time will tell…

[A footnote that after a long discussion Java decided not to implement a var/val keyword pair as in Scala. val would define an immutable value – whilst valuable it was decided this would over-complicate type inference, and it is arguably not all that useful on local variables anyway. That’s a bit disappointing but it’s true that immutability is orthogonal to type inference. I hope that a future version of Java will introduce some strong language level support for immutability.]

Trying Java 10 in Eclipse

If you’re keen to try Java 10 (of course you are!) then you can download the JDK from Oracle at the usual place.

I’m still with Eclipse (for the time being at least), and to use Java 10 (at the time of writing), you need to be on the latest version (currently Oxygen.3 aka 4.7.3) and you need to install (via the Marketplace) a Java 10 plugin.

Java 10 on the Marketplace

After installing the plugin, restart Eclipse and then define a JDK (using the usual tortuous Window -> Preferences -> Java -> Installed JREs and navigate to the folder you downloaded the JDK to).If you haven’t installed the plugin, this will fail because eclipse won’t recognise the new structure and layout of the JDK.

Now you can create a new project – I haven’t changed my default compiler compliance level so I’m getting a promising looking message saying the compliance level is going to be changed for this project…

New Project Window – looks like the compliance level is going to be set properly!

And that’s it. You can now create a class and start using the new features – the next blog post will explore what I think is the best of them, Local Variable Type Inference.

[edit1: I’m far too Eclipse-centric, I do need to address this soon. I can’t see any sign of Java 10 support in NetBeans but JetBrains/IntelliJ as usual are well ahead of the curve, they announced support for Java 10 as far back as last November. ]

[edit2: JetBrains are running a live Java 10 Webinar hosted by Trisha Gee on Thursday, Apr 5 2018, 4:00 PM – 5:00 PM CEST, check it out if you can.]

Java 10 is here…already!

Wow – that was quick. After years of interminable delays to Java 9, all of a sudden the new strict release cycle has kicked in and the first result of that, Java 10 was released on March 20 2018.

In brief, there will be a new release of Java every 6 months from now on and every three years there will be a “long term support” release. Java 11 (due late 2018) is planned to be the first LTS. (Thanks to David Morgan in the comments for spotting my error in early versions of this post!)

Obviously many projects/organizations will insist on only using the LTS releases, so the reality for many Java developers is that adopting Java 11 will probably be long term stretch goal. (By my reckoning, if they keep their promise, Java 17 will be the next LTS after that – it will interesting psychological challenge for the likes of banks to upgrade from 11 to 17 in one jump!).

For anyone not tied to LTS releases, we’re going to get a new set of toys to play with every six months!

Here’s the full set of JEPs (JDK Enhancement Proposals) you can use right now in Java 10. Most of them are under the hood enhancements, cleanups and non-coding related changes. But the headline for Java programmers is the first of them, JEP286 – which I’ll write about in my next blog post. It’s a stunning improvement to Java.

286: Local-Variable Type Inference
296: Consolidate the JDK Forest into a Single Repository
304: Garbage-Collector Interface
307: Parallel Full GC for G1
310: Application Class-Data Sharing
312: Thread-Local Handshakes
313: Remove the Native-Header Generation Tool (javah)
314: Additional Unicode Language-Tag Extensions
316: Heap Allocation on Alternative Memory Devices
317: Experimental Java-Based JIT Compiler
319: Root Certificates
322: Time-Based Release Versioning

Hello, is this thing on?

A year’s gap from blogging – bad Richard.

Blogging sort of tailed off, partly as a result of the ancient and depressing blog platform I was using (every post needed the editing of raw 1990’s style html – <hr> tags anyone?) but also due to a lot of activity and general busy-ness at VirtualPairProgrammers.

In the last year I’ve released courses on:

Plus as a substitute for blogging, I’ve been a regular guest on the All things Java Podcast which was fun but apparantly no-one listens to it. [Edit: the podcasts will continue and I hope I’ll still be a guest on them, but I want to remind myself that blogging is just as if not more important than podcasts]. So it’s back to blogging and more new courses are in the pipeline.