Akka supports dispatchers for both event-driven lightweight threads and thread-based Actors. For thread-based Actors each dispatcher is bound to a dedicated OS thread.

Default dispatcher is a single event-based dispatcher for all Actors created. The dispatcher used is this one:

Dispatchers.globalExecutorBasedEventDrivenDispatcher

For many cases it becomes mandatory to group Actors together for a dedicated dispatcher, then we can override the defaults and define our own dispatcher.

Setting the Dispatcher
Normally we set the dispatcher in Actor itself

class EchoActor extends Actor {
     self.dispatcher = ..... // set the dispatcher
}

actorRef.dispatcher = dispatcher

There are different kind of dispatchers

• Thread-based
• Event-based
• Priority event-based
• Work-stealing

Thread-based
It binds dedicated OS thread to each Actor. The messages are posted to LinkedBlockingQueue which feeds messages to dispatcher one by one. It has worst performance and scalability. We also cannot share it among actors. Although Actors do not block for threads in this case.
Code example

class EchoActor extends Actor {
     self.dispatcher = Dispatchers.newThreadBasedDispatcher(self)
     ....
}

Event-based
The ExecutorBasedEventDrivenDispatcher binds a set of Actors to a thread pool backed up by a BlockingQueue. The dispatcher must be shared among Actors. This dispatcher is highly configurable and here we can specify things like ‘type of queue’, ‘max items’ , ‘rejection-policy’.
Code example

class EchoActor extends Actor {
self.dispatcher = Dispatchers.newExecutorBasedEventDrivenDispatcher(name)
 .withNewThreadPoolWithLinkedBlockingQueueWithCapacity(100)
 .setCorePoolSize(16)
 .setMaxPoolSize(128)
 .setKeepAliveTimeInMillis(60000)
 .setRejectionPolicy(new CallerRunsPolicy)
 .build
 ....
}

Priority event-based
It is meant for handling messages when priorities are assigned to messages. It is done by using PriorityExecutorBasedEventDrivenDispatcher. It requires a PriorityGenerator as an attribute in its constructor.

Let’s look at an example where we have a PriorityExecutorBasedEventDrivenDispatcher used for a group of messages fired on an actor.

package com.meetu.akka.dispatcher

import akka.actor.Actor.actorOf
import akka.actor.Actor
import akka.dispatch.PriorityExecutorBasedEventDrivenDispatcher
import akka.dispatch.PriorityGenerator

object PriorityDispatcherExample extends App {
  val actor = Actor.actorOf(
    new Actor {
      def receive = {
        case x => println(x)
      }
    })

  val priority = PriorityGenerator {
    case "high priority" => 0
    case "low priority" => 100
    case _ => 50
  }

  actor.dispatcher = new PriorityExecutorBasedEventDrivenDispatcher("foo", priority)

  actor.start
  actor.dispatcher.suspend(actor)

  actor ! "low priority"
  actor ! "others"
  actor ! "low priority"
  actor ! "high priority"

  actor.dispatcher.resume(actor)
  Actor.registry.shutdownAll
}

If we execute the code, high-priority messages are served first even when low-priority fired before the high-priority messages. This is how the output will appear if executed by sbt on command line:

Work-stealing
‘ExecutorBasedEventDrivenWorkStealingDispatcher’ is one of my favorite dispatcher. It redistributes work to actors that use the same dispatcher and that do not currently have any messages the mailbox. It is a great way to increase performance of the system.

Usual way to use it is to create an Actor companion object to hold the dispatcher and then set it in Actor explicitly.

Lets look at the code example which uses this dispatcher.

package com.meetu.akka.dispatcher

import akka.actor.Actor
import akka.actor.Actor._
import akka.dispatch.Dispatchers

object WorkStealingDispatcherExample extends App {
  val actor = actorOf[SimpleActor]
  actor.start
  actor ! "Hello"
  Actor.registry.shutdownAll
}

object SimpleActor {
  val dispatcher = Dispatchers.newExecutorBasedEventDrivenWorkStealingDispatcher("executordispatcher").build
}

class SimpleActor extends Actor {
  self.dispatcher = SimpleActor.dispatcher
  def receive = {
    case msg => println(msg)
  }
}

Now lets have a look at a code listing for ExecutorDispatcherApplicationExample. It’s constituents are a Scala Object ExecutorDispatcherApplicationExample. Two Pairs of Scala Class and Object (Object is there to share dispatchers) and last three case classes which corresponds to message passing between Actors. Functionally ProcessorUsingExecutorBasedDispatcher is a master actor as it spawns workers and ProcessorUsingExecutorBasedDispatcherWorker is a worker.

package com.meetu.akka.dispatcher

import akka.actor.Actor.actorOf
import akka.actor.Actor
import akka.dispatch.Dispatchers
import akka.routing.CyclicIterator
import akka.routing.Routing

object ExecutorDispatcherApplicationExample extends App {
  val processorUsingExecutorBasedDispatcher = actorOf[ProcessorUsingExecutorBasedDispatcher]
  processorUsingExecutorBasedDispatcher.start
  processorUsingExecutorBasedDispatcher ! new SimpleMessage("Hello Actor!!")
}

object ProcessorUsingExecutorBasedDispatcher {
  val dispatcher = Dispatchers.newExecutorBasedEventDrivenDispatcher("ed1").setCorePoolSize(10).setMaxPoolSize(10).build
}

class ProcessorUsingExecutorBasedDispatcher extends Actor {
  self.dispatcher = ProcessorUsingExecutorBasedDispatcher.dispatcher
  var startTime: Long = -1
  var parallelCounter: Int = 1000

  def receive = {
    case SimpleReply => {
      parallelCounter -= 1
      if (parallelCounter == 0) {
        val endTime = System.currentTimeMillis
        println("Total Time:: " + (endTime - startTime))
        Actor.registry.shutdownAll
      }
    }

    case SimpleMessage(msg) =>
      startTime = System.currentTimeMillis
      val workers = Vector.fill(15)(actorOf[ProcessorUsingExecutorBasedDispatcherWorker].start)
      val router = Routing.loadBalancerActor(CyclicIterator(workers)).start
      for (i <- 0 until parallelCounter) {
        router ! new SimpleRequest(msg)
      }
  }
}

object ProcessorUsingExecutorBasedDispatcherWorker {
  val dispatcher = Dispatchers.newExecutorBasedEventDrivenDispatcher("ed2").setCorePoolSize(10).setMaxPoolSize(10).build
}

class ProcessorUsingExecutorBasedDispatcherWorker extends Actor {
  self.dispatcher = ProcessorUsingExecutorBasedDispatcherWorker.dispatcher
  def receive = {
    case SimpleRequest(msg) => {
      val sleepTime = java.lang.Math.random() * 510
      Thread.sleep(sleepTime.toLong)
      self reply SimpleReply
    }
  }
}

The Application ExecutorDispatcherApplicationExample starts the ProcessorUsingExecutorBasedDispatcher Actor and passes a SimpleMessage to it.

ProcessorUsingExecutorBasedDispatcher Actor uses ExecutorBasedDispatcher and it does so by using a companion object. Please notice that the dispatcher of the object is assigned to self.

...
self.dispatcher = ProcessorUsingExecutorBasedDispatcher.dispatcher
...

The receiving block in ProcessorUsingExecutorBasedDispatcher “case SimpleMessage(msg) =>” starts the workers and a Router over it. It also finally fires “parallelCounter” number of messages to the router.

The Worker Actor ProcessorUsingExecutorBasedDispatcherWorker also uses ExecutorBasedDispatcher. Its received block “case SimpleRequest(msg) =>” replies back to ProcessorUsingExecutorBasedDispatcher after sleeping off for a while.

The Actor ProcessorUsingExecutorBasedDispatcher receives the response and it continue to receive it till all responses are received from Worker Actor. Finally it calculates total time elapsed for parallel process execution.

We also can code this using Work Stealing Dispatcher. The complete code example is given below:

package com.meetu.akka.dispatcher

import akka.actor.Actor.actorOf
import akka.actor.Actor
import akka.dispatch.Dispatchers
import akka.routing.CyclicIterator
import akka.routing.Routing

object WorkStealingDispatcherApplicationExample extends App {
  val processorUsingWorkStealingDispatcher = actorOf[ProcessorUsingWorkStealingDispatcher]
  processorUsingWorkStealingDispatcher.start
  processorUsingWorkStealingDispatcher ! new SimpleMessage("Hello Actor!!")
}

object ProcessorUsingWorkStealingDispatcher {
  val dispatcher = Dispatchers.newExecutorBasedEventDrivenWorkStealingDispatcher("wsd1").setCorePoolSize(10).setMaxPoolSize(10).build
}

class ProcessorUsingWorkStealingDispatcher extends Actor {
  self.dispatcher = ProcessorUsingWorkStealingDispatcher.dispatcher
  var startTime: Long = -1
  var parallelCounter: Int = 1000

  def receive = {
    case SimpleReply => {
      parallelCounter -= 1
      if (parallelCounter == 0) {
        val endTime = System.currentTimeMillis
        println("Total Time:: " + (endTime - startTime))
        Actor.registry.shutdownAll
      }
    }

    case SimpleMessage(msg) =>
      startTime = System.currentTimeMillis
      val workers = Vector.fill(15)(actorOf[ProcessorUsingWorkStealingDispatcherWorker].start)
      val router = Routing.loadBalancerActor(CyclicIterator(workers)).start
      for (i <- 0 until parallelCounter) {
        router ! new SimpleRequest(msg)
      }
  }
}

object ProcessorUsingWorkStealingDispatcherWorker {
  val dispatcher = Dispatchers.newExecutorBasedEventDrivenWorkStealingDispatcher("wsd2").setCorePoolSize(10).setMaxPoolSize(10).build
}

class ProcessorUsingWorkStealingDispatcherWorker extends Actor {
  self.dispatcher = ProcessorUsingWorkStealingDispatcherWorker.dispatcher
  def receive = {
    case SimpleRequest(msg) => {
      val sleepTime = java.lang.Math.random() * 510
      Thread.sleep(sleepTime.toLong)
      self reply SimpleReply
    }
  }
}

case class SimpleRequest(msg: String)

case class SimpleMessage(msg: String)

case class SimpleReply

That is all about dispatchers, they are a powerful way of managing performance, throughput and scalability of the system. All code examples are here at my github repository. It is a sbt project. The command

sbt run

will ask for the program to run, just enter the number and see how it executes.

Most of the times a bunch of Akka actors are required for a common functionality; therefore we do tend to manage them as a group.

This is where Akka Supervisor comes into the picture. Akka Supervisors are responsible for starting, stopping and monitoring child Akka actors.

Lets look now how to use supervisor over Akka Actors. Just to keep things simple we have an Actor which simply echoes the message it receives. Here is the code listing for EchoActor:

/**
 * An Actor that echoes everything you send to it
 */
class EchoActor extends Actor {
  def receive = {
    case msg => {
      println(msg)
    }
  }
}

We will build supervision on a bunch of EchoActor, lets say our supervisor is EchoActorSupervisor. But before we actually look at its code listing we understand its constituent elements first.

The Supervisor first require SupervisorConfig. This SupervisorConfig in turn requires fault handling strategy for the group of actors. They are:

• AllForOneStrategy
• OneForOneStrategy

The AllForOne fault handler will restart all the components and OneForOneStrategy fault handler will restart just the component which died. Fault Handler takes as parameter a List of Exceptions which will be handled, maximum number of restart tries and within-time in millis

Lets have a look how does our SupervisorConfig looks like:

...
val supervisorConfig = SupervisorConfig(OneForOneStrategy(List(classOf[Exception]), 3, 500), Nil)
...

Supervisor can now be created by passing in SupervisorConfig created in previous listing:

...
val supervisorConfig = SupervisorConfig(OneForOneStrategy(List(classOf[Exception]), 3, 500), Nil)
val supervisor = Supervisor(supervisorConfig)
...

now we can link and unlink actors to the supervisor. In our case since we are using EchoActor we can link two Actor Reference to the supervisor. In the code below we are linking and starting two EchoActor.

...
val echoActor1 = actorOf[EchoActor]
val echoActor2 = actorOf[EchoActor]

val supervisorConfig = SupervisorConfig(OneForOneStrategy(List(classOf[Exception]), 3, 500), Nil)
val supervisor = Supervisor(supervisorConfig)

supervisor.link(echoActor1)
supervisor.link(echoActor2)

echoActor1.start
echoActor2.start
...

If you like a shorthand way of doing it where you can start and link the actors in a go, here is the code listing for that.

...
val echoActor1 = actorOf[EchoActor]
val echoActor2 = actorOf[EchoActor]

val supervisorConfig = SupervisorConfig(AllForOneStrategy(List(classOf[Exception]), 3, 1000),
    Supervise(echoActor1, Permanent) :: Supervise(echoActor2, Permanent) :: Nil)
val supervisor = Supervisor(supervisorConfig)
...

Since, we have provided Fault Tolerance Strategy we will also like to see it working in action. For that Echo Actor should die after throwing an Exception.

Let Echo Actor be changed to throw an Exception when it has an ErrorInducingMessage. For this to occur we will introduce a simple case class ErrorInducingMessage and we will change EchoActor to die when it gets this message. Here I also override a lifecycle method postRestart in EchoActor to show that actor came up after throwing exception.

The code listing for described changes:

/**
 * An Actor that echoes everything you send to it
 */
class EchoActor extends Actor {

  def receive = {
    case ErrorInducingMessage => throw new Exception("Dying due to error inducing message")
    case msg => println(msg)
  }
  
  override def postRestart(err:Throwable) = {
    println("I am back again...")
  }
}

case class ErrorInducingMessage

To show the fault handling strategy in action we will send one message each to actors. Then we send one ErrorInducingMessage to one of the actor and finally we send one message each again. Lets combine all of this together.

package com.meetu.akka

import akka.actor.Actor.actorOf
import akka.actor.Actor
import akka.actor.Supervisor
import akka.config.Supervision.OneForOneStrategy
import akka.config.Supervision.SupervisorConfig

object Application extends App {

  val echoActor1 = actorOf[EchoActor]
  val echoActor2 = actorOf[EchoActor]
  val supervisorConfig = SupervisorConfig(OneForOneStrategy(List(classOf[Exception]), 3, 500), Nil)
  val supervisor = Supervisor(supervisorConfig)

  supervisor.link(echoActor1)
  supervisor.link(echoActor2)

  echoActor1.start
  echoActor2.start
  
  echoActor1 ! "Hello actor 1"
  echoActor2 ! "Hello actor 2"
  
  echoActor1 ! ErrorInducingMessage
  Thread.sleep(1000)
  echoActor1 ! "Hello actor 1"
  echoActor2 ! "Hello actor 2"
}

/**
 * An Actor that echoes everything you send to it
 */
class EchoActor extends Actor {

  def receive = {
    case ErrorInducingMessage => throw new Exception("Dying due to error inducing message")
    case msg => println(msg)
  }
  
  override def postRestart(err:Throwable) = {
    println("I am back again...")
  }
}

case class ErrorInducingMessage

For want of shorter code where we do not explicitly start the actors here is the complete code listing.

package com.meetu.akka

import akka.actor.Actor.actorOf
import akka.actor.Actor
import akka.actor.Supervisor
import akka.config.Supervision.AllForOneStrategy
import akka.config.Supervision.Permanent
import akka.config.Supervision.Supervise
import akka.config.Supervision.SupervisorConfig

object Application extends App {

  val echoActor1 = actorOf[EchoActor]
  val echoActor2 = actorOf[EchoActor]

  val supervisorConfig = SupervisorConfig(AllForOneStrategy(List(classOf[Exception]), 3, 1000),
    Supervise(echoActor1, Permanent) :: Supervise(echoActor2, Permanent) :: Nil)
  val supervisor = Supervisor(supervisorConfig)

  echoActor1 ! "Hello actor 1"
  echoActor2 ! "Hello actor 2"

  echoActor1 ! ErrorInducingMessage
  Thread.sleep(1000)
  echoActor1 ! "Hello actor 1"
  echoActor2 ! "Hello actor 2"
}

/**
 * An Actor that echoes everything you send to it
 */
class EchoActor extends Actor {

  def receive = {
    case ErrorInducingMessage => throw new Exception("Dying due to error inducing message")
    case msg => println(msg)
  }

  override def postRestart(err: Throwable) = {
    println("I am back again...")
  }
}

case class ErrorInducingMessage

When we run this we see that there are two messages from each actors. There is some error trace as expected. Then we see that one of the actor is back again. Finally we see that two messages are printed by each actors.

Fault tolerance is important in the sense that we are sure of whether all actors managed by a supervisor are dead or none of them are dead. Instead of preventing from things to go wrong we think about recovery from a crash. This in sense encourages non-defensive programming.

The Akka code for this can be downloaded from here. This is a sbt project and code can be executed by executing following at command prompt

 sbt run

I tried to create AKKA project with latest SBT but got stuck. Old SBT used to ask to create a new project in case it did not find any in the directory. With new SBT it is not the case. If you want to know how to go about creating new Akka project with SBT read on.

After installing SBT if we type in sbt in the command prompt in an empty directory, this is what we are likely to see.

In order to create project execute the following commands in the sbt session.

> set name := "AkkaQuickStart"
> set version := "1.0"
> set scalaVersion := "2.9.0-1"
> session save
> exit

We should get the following output if we type the above mentioned commands in sbt session.

SBT creates files and directories when we executed the commands. It creates build.sbt and it contains the same values we typed in sbt session. Other directories like target and project are of little consequence to us.

Project directory will become important later when we will try to add sbteclipse plugin. My project directory contains the following subdirectories and files.

This is how my my build.sbt files looks like after executing the commands in sbt session

name := "AkkaQuickStart"

version := "1.0"

scalaVersion := "2.9.0-1"

One thing that has also changed from the past is that the src directories are not created by default. I used the sbteclipse plugin to do it for me.

Create directory plugins inside project and add build.sbt to it. Add following contents to it:

resolvers += {
  val typesafeRepoUrl = new java.net.URL("http://repo.typesafe.com/typesafe/releases")
  val pattern = Patterns(false, "[organisation]/[module]/[sbtversion]/[revision]/[type]s/[module](-[classifier])-[revision].[ext]")
  Resolver.url("Typesafe Repository", typesafeRepoUrl)(pattern)
}

libraryDependencies <<= (libraryDependencies, sbtVersion) { (deps, version) => 
  deps :+ ("com.typesafe.sbteclipse" %% "sbteclipse" % "1.3-RC1" extra("sbtversion" -> version))
}

Execute the following command.

sbt "eclipse create-src"

It creates common source directories like src/main/scala, src/main/test, etc and also eclipse project files.

The project is ready to imported in Eclipse IDE. We also require Akka dependencies in our project. This can be done by adding resolvers and library dependencies to build.sbt at the top level. Add the following entries in build.sbt:

name := "AkkaQuickStart"

version := "1.0"

scalaVersion := "2.9.0-1"

resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"

libraryDependencies ++= Seq(
  "se.scalablesolutions.akka" % "akka-actor" % "1.1.3",
  "se.scalablesolutions.akka" % "akka-typed-actor" % "1.1.3",
  "se.scalablesolutions.akka" % "akka-amqp" % "1.1.3",
  "se.scalablesolutions.akka" % "akka-testkit" % "1.1.3"
)

Whenever we add new dependencies we must execute “sbt eclipse” to regenerate the eclipse project configuration files. Now in order that code for Akka in our projects compiles correctly in Eclipse we execute the following command:

sbt eclipse

We have a skeleton project with all dependencies and build scripts ready for an AKKA project. Lets create a simple Echo Akka Actor to validate that our configurations work correctly.

The object Application starts EchoActor passes message to it and then stops it. EchoActor prints the message received from Application.

package com.meetu.akka

import akka.actor.Actor.actorOf
import akka.actor.Actor

object Application extends App {

  val actor = actorOf(new EchoActor)
  actor.start
  actor ! "Hello Akka Echo Actor"
  actor.stop
}

/**
 * An Actor that echoes everything you send to it
 */
class EchoActor extends Actor {
  def receive = {
    case msg => {
      println(msg)
    }
  }
}

If all is fine the code will compile without any issues but before we run this Application as scala application we need to download Akka from here and set AKKA_HOME in PATH variable.

After AKKA_HOME points to the right directory we can run the application either from Eclipse or from SBT. For Eclipse run it as Scala Application. For SBT use the command provided below. You should see the “Hello Akka Echo Actor” message in the console.

sbt "run-main com.meetu.akka.Application"

That is all we need for creating Akka project using the latest SBT. We now know how to create an Akka project using SBT and also to add dependencies to it. We can also generate Eclipse IDE specific project files using sbteclipse.

I have created a sample project AkkaExperiments which has the code for starting Akka project using SBT.

Today I happened to have a look at it on Slideshare. Looking at the number of views it seems to be doing good. Out of my previous employer presentations its having highest hits. Scala is really doing good in terms of popularity.

Presentation is still old. To get started concepts are still the same. Enjoy the presentation…

Sessions at the conference dealt with key topics like Cloud Security, Amazon Elastic Beanstalk, Legal Issues in Cloud Computing, OpenStack, Xen Cloud Platform, Rails and CouchDB on the cloud, CloudFoundry, Gigapaces PAAS, Monitoring Cloud Applications, ORM with Objectify-Appengine, Scalable Architecture on Amazon AWS Cloud, Cloud Lock-in, Cloud Interoperability, Apache Hadoop, Map Reduce and Predictive Analysis.

My talk focussed on managing persistence on GAE. It dealt with choices available to a developer and then focussed on doing it with Objectify-Appengine.

Demo application is present here for download. For the instructions on how to run it please read the wiki.

Scala collection is elegant and concise. Scala Collections like java is object-oriented and we can work with generic types. It is optionally persistent i.e can be mutable and immutable. It provides higher order methods like foreach, map and filter.

Scala collections follow uniform return type principle. Which basically means that when you perform an operation on a collection they return a collection of the same type.

The root Trait in Scala collections is Traversable. It may take some time to get used to it as Scala collections root Traits like Traversable have a big bunch of methods.

The for notation is more readable. It is basically like a for loop with a return type. Look for yourself how readable it is compared to the original code.

This code listing is without a for notation

scala> val listOfNumbers = List(1, 2, 3)
listOfNumbers: List[Int] = List(1, 2, 3)

scala> val flattenedListOfNumbers = listOfNumbers flatMap (0 to _)
flattenedListOfNumbers: List[Int] = List(0, 1, 0, 1, 2, 0, 1, 2, 3)

After using for notation

scala> val listOfNumbers = List(1, 2, 3)
listOfNumbers: List[Int] = List(1, 2, 3)

scala> val flattenedList = for(number <- listOfNumbers; range <- 0 to number) yield range
flattenedList: List[Int] = List(0, 1, 0, 1, 2, 0, 1, 2, 3)

One more interesting thing we can do with a Map is that we can reverse key and value with the following code

scala> val aMap = Map("India" -> "Delhi", "France" -> "Paris", "Italy" -> "Rome")
aMap: scala.collection.immutable.Map[java.lang.String,java.lang.String] = Map(India -> Delhi, France -> Paris, Italy -> Rome)

scala> val reverseMap = aMap map{case(k, v) => (v, k)}
reverseMap: scala.collection.immutable.Map[java.lang.String,java.lang.String] = Map(Delhi -> India, Paris -> France, Rome -> Italy)

scala> val alsoReverseMap = for((k, v) <- aMap) yield (v -> k) 
alsoReverseMap: scala.collection.immutable.Map[java.lang.String,java.lang.String] = Map(Delhi -> India, Paris -> France, Rome -> Italy)

Reference were from Martin’s Odersky talk on Future Proofing Collections and Scala Collections API documentation. Enjoy the presentation …

Let’s have a look at a simple JUnit4 Test in Scala Test

package com.inphina.ibat.junit

import org.scalatest.junit.AssertionsForJUnit
import org.junit.Assert._
import org.junit.Test
import  org.junit.Before

class SimpleJunit4Demo extends AssertionsForJUnit {

  var sb: StringBuilder = _

  @Before def initialize() {
    sb = new StringBuilder("Welcome To ")
  }

  @Test
  def verifyEasy() {
    sb.append("ScalaTest!")
    assertEquals("Welcome To ScalaTest!", sb.toString)
  }

}

This Test implementation is not very different from a normal JUnit Test we are accustomed to. In order to understand ScalaTest there are three concepts we need to be aware of.

• Suite: A collection of tests. A Test is anything that has a name and can succeed or fail
• Runner: ScalaTest provides a Runner application that can run Suites of Tests
• Reporter: As Tests are run events are fired to a reporter, it takes care of presenting results back to the user

Let’s have a bit more detailed look at the Suite which is a Trait.

When you run a Test in Scala Test you basically invoke run(Option[String], Reporter, .. ) on Suite Object.

• It then calls runNestedSuites(Reporter, ..).
• And it calls runTests(Option[String], Reporter, ..)

runNestedSuites(Reporter, ..) then invokes nestedSuites() : List[Suite] to get a List of nested Suites. Which are required for running tests.

runTests(Option[String], Reporter, ..) calls def testNames: Set[String] to get a Set of test names it needs to run. For each test name it calls runTest(Reporter, …) It wraps the test code as a Function Object with a name and passes it to withFixture(NoArgTest) which actually runs the test.

Way of working with Scala Test is working with Traits. Just like any other Trait we can do Mixin composition to have a different flavor of Tests we want to. Scala Test have some core Traits and then we can mix it with other available Traits.

Core Traits

Mix In other Traits

So, essentially working with Scala Test requires picking up a Core Trait and mixin it with other provided Traits. Some of the Core Traits are

• Suite
• FunSuite
• Spec
• FlatSpec
• WordSpec
• FeatureSpec

Let’s have a look at a Suite Example:
Traits approach to writing tests. Simply create classes extending Suite and define test methods. Test methods have names testXXXX. All methods must be public. Scala Test provides === operator. It is defined in Traits Assertions. Allows the failure report to include both right and left values.

package com.inphina.ibat.suite

import org.scalatest.Suite

/**
 * Traits approach to writing tests. Simply create classes
 * extending Suite and define test methods.
 *
 * Test methods have names testXXXX. All methods must be public.
 *
 * Scala Test provides === operator. It is defined in Traits Assertions. Allows the failure report
 * to include both right and left values.
 */

class SimpleSuiteDemo extends Suite {

   def testAddition() {
        val sum = 1 + 1
        assert(sum === 3)
   }

  def testSubtraction() {
    val diff = 4 - 1
    assert(diff === 3)
  }
}

If we want to have some Mixin with other Traits in Scala Test then we can use ShouldMatchers. It provides DSL type of assertions and makes the code much more readable. Let’s have a look at the code listing below.

package com.inphina.ibat.suite

import org.scalatest.Suite
import org.scalatest.matchers.ShouldMatchers

/**
 * Mixin Traits ShouldMatchers for DSl type assertions
 */

class DSLMatchersSuiteDemo extends Suite with ShouldMatchers {

  def testAddition() {
    val sum = 1 + 1
    sum should equal (2)
    sum + 2 should equal (4)
  }

}

Both the Tests till now are not purely functional Tests. They resemble normal Java Tests use FunSuite Trait to write Functional Scala Tests.
FunSuite

• “test” is a method defined in FunSuite Trait. Test name goes in parentheses and the test code goes in curly braces
• The test code in curly braces is passed as a by-name parameter to “test” method which registers for later execution
• A FunSuite’s life-cycle has two phases: the registration phase and the ready phase

Let’s have a look at the code example

package com.inphina.ibat.funsuite

import org.scalatest.FunSuite

/**
 * For writing Functional Tests use FunSuite Fun => Functional Suite. "test" is a method defined in FunSuite Trait.
 * Test name goes in parentheses and the test code goes in curly braces. One benefit of using FunSuite compared to
 * Suite is that we need not name the test with testXXXX style names.
 *
 * The test code in curly braces is passed as a by-name parameter to "test" method which registers for later execution
 *
 * More benefits we can easily give long names in tests instead of camel case notation generally used.
 *
 * A FunSuite's life-cycle has two phases: the registration phase and the ready phase. It starts in registration phase
 * and enters ready phase the first time run is called on it. It then remains in ready phase for the remainder of its
 * lifetime.
 */

class SimpleFunSuiteDemo extends FunSuite {

  test("addition of numbers") {
    val sum = 1 + 2
    assert(sum === 2)
  }

  test("subtraction") {
    val diff = 5 - 2
    assert(diff === 3)
  }

}

One of the immediate benefits is that we can use more descriptive test names instead of Camel case method names we are so used to.

We have looked at Suite and FunSuite way of writing tests now lets have a look at the FeatureSpec which is a higher level way of writing Tests. It basically represents a Suite of Tests where each Test represent a scenario of a Feature.

A FeatureSpec contains feature clauses and scenarios. Feature Clause and Scenario are defined by “feature” and “scenario”. Both of them are defined as methods in FeatureSpec.
Let’s have a look at an example of FeatureSpec mixin with GivenWhenThen Trait.

package com.inphina.ibat.featurespec

import org.scalatest.FeatureSpec
import org.scalatest.GivenWhenThen
import scala.collection.mutable.Stack


class StackFeatureSpec extends FeatureSpec with GivenWhenThen {

  feature("The user can pop an element off the top of the stack") {

    info("As a programmer")
    info("I want to be able to pop items off the stack")
    info("So that I can get them in last-in-first-out order")

    scenario("pop is invoked on a non-empty stack") {

      given("a non-empty stack")
      val stack = new Stack[Int]
      stack.push(1)
      stack.push(2)
      val oldSize = stack.size

      when("when pop is invoked on the stack")
      val result = stack.pop()

      then("the most recently pushed element should be returned")
      assert(result === 2)

      and("the stack should have one less item than before")
      assert(stack.size === oldSize - 1)
    }

    scenario("pop is invoked on an empty stack") {

      given("an empty stack")
      val emptyStack = new Stack[String]

      when("when pop is invoked on the stack")
      then("NoSuchElementException should be thrown")
      intercept[NoSuchElementException] {
        emptyStack.pop()
      }

      and("the stack should still be empty")
      assert(emptyStack.isEmpty)
    }
  }
}

Major benefits by Scala Test are similar to scala itself. It is concise, provides you a flexible manner of testing your application be it Scala or Java. Works with popular Java frameworks like TestNg and JUnit and EasyMock. And last but not the least provides a nice way to introduce Scala in the project.

I presented it on iBat session at Inphina. Content is nearly the same, enjoy the presentation …

We have a domain Employee :

@Entity
public class Employee {

  @Id
  @GeneratedValue(strategy = GenerationType.AUTO)
  private Long id;

  private String name;

  // … methods omitted
}

We also have a EmployeeRepository handling the Employee entities. The interface and the implementation are given below.

package com.inphina.springdata.repository;

public interface EmployeeRepository {
	
	Employee save(Employee employee);
	
	List<Employee> findAll();
	
	Employee findById(Long id);
	
	List<Employee> findByName(String name, int page, int pageSize);

}

@Repository
@Transactional(readOnly = true)
public class EmployeeRepositoryImpl implements EmployeeRepository {
	
	@PersistenceContext
	private EntityManager entityManager;

	@Override
	@Transactional
	public Employee save(Employee employee) {
		if(employee.getId() == null) {
			entityManager.persist(employee);
		} else {
			entityManager.merge(employee);
		}
		return employee;
	}

	@Override
	public List<Employee> findAll() {
		return entityManager.createQuery("select e from Employee e", Employee.class).getResultList();
	}

	@Override
	public Employee findById(Long id) {
		return entityManager.find(Employee.class, id);
	}

	@Override
	public List<Employee> findByName(String name, int page, int pageSize) {
		TypedQuery<Employee> query = entityManager.createQuery("select e from Employee e where e.name = ?1", Employee.class);
		query.setParameter(1, name);
		query.setFirstResult(page * pageSize);
		query.setMaxResults(pageSize);
		return query.getResultList();
	}
}

the client need to decide whether to call merge(…) or persist(…) on the EntityManager. We use the id-field of the Employee to decide whether we consider an Employee object as new or not.

Spring Data JPA provides a repository programming model that starts with an interface per managed domain object. That means for the Employee domain there is an Interface such as EmployeeRepository .

@Transactional(readOnly = true)
public interface EmployeeRepository extends JpaRepository<Employee, Long> { ... }

by extending JpaRepository we get generic CRUD methods into our type that allows working with Employees. This also allow the Spring Data JPA repository infrastructure to scan the classpath for this interface and create a Spring bean for it.

To have Spring create a bean that implements this interface, we need to use the Spring JPA namespace and activate the repository support using the appropriate element:

<jpa:repositories base-package="com.inphina.springdata" />

This scans all the package below com.inphina.springdata for interfaces extending JpaRepository and creates a spring bean backed by an implementation of SimpleJpaRepository

Now lets use a bit of Spring Data help in reducing the EmployeeRepositoryImpl code. We change the interface EmployeeRepository to this:

public interface EmployeeRepository extends JpaRepository<Employee, Long>{
	Employee save(Employee employee);

	List<Employee> findByName(String name, int page, int pageSize);
}

The EmployeeRepositoryImpl can then implement findByName(String name, int page, int pageSize) and Employee save(Employee employee);
as shown in the code below.

@Transactional(readOnly = true)
public class EmployeeRepositoryImpl implements EmployeeRepository {
	
	@Autowired
	private EmployeeRepository repository;

      @Override
	public Employee save(Employee employee) {
		return repository.save(employee);
	}

      @Override
	public List<Employee> findByName(String name, int page, int pageSize) {
		return repository.findByName(name, page, pageSize);
	}
}

Here we simply delegate the call to save(…) to the repository. By default the repository implementation considers an entity new if its id property is null similar to the first code listing of EmployeeRepositoryImpl

Spring Data JPA provides an abstraction consisting of two interfaces: Pageable (to capture pagination request information) as well as Page (to capture the result as well as meta-information). We can add the findByName in the interface itself. The need for the interface implementation is not required now. This is how the interface will look after the changes.

@Transactional(readOnly = true)
public interface EmployeeRepository extends JpaRepository<Employee, Long> {

	/**
	 * Returns a Page of {@link Employee}s with the given name.
	 * @param name
	 * @param page
	 * @param pageSize
	 * @return
	 */
	Page<Employee> findByName(String name, int page, int pageSize);
}

To summarize we have seen that there is significant reduction of the code. After refactoring what remains is one interface and a XML declaration. Spring Data JPA does looks similar to Hades where simple queries can be generated from method names. However we might come to see integration of QueryDsl in near future.

How can we use this @Configuration in our application? and should we use it when we have XML and annotations for configuration already?

In Spring 2.5 we find that there is a support for Property Externalization. Which was good in several ways than before. What it meant was, that the properties file could be modified for different build environments by the administrator.

One good thing about this approach is that the administrator need not edit the application context file. Editing verbose XML has plenty of room for errors. Editing the properties file on the other hand is a whole lot simpler and is less error prone.

For example for configuring the datasource this is what we do typically. We used Property externalization in Spring 2.5 using PropertyPlaceHolderConfigurer.

<context:property-placeholder location="classpath:datasource.properties" />
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
		<property name="driverClassName"><value>${dataSource.driverClassName}</value></property>
		<property name="url"><value>${dataSource.url}</value></property>
		<property name="username"><value>${dataSource.username}</value></property>
		<property name="password"><value>${dataSource.password}</value></property>
</bean>

Important point to note here is that it is done at the configuration time. So, it is only done once while the application context file loads up.

This approach does have some drawbacks though:

• It only supports properties
• Replacement is done at initialization time not at bean creation time
• It does not support conditionals or other constructs
• It is not Typesafe

Let’s have a look how can we use a java based Spring configuration.

First we create a Java class and add @Configuration annotation to it. Then we define methods to instantiate the application’s classes and call other methods to configure the dependencies. We also add @Bean annotation to these methods.

For example If we have a bean name MyBean then we can create a Configuration annotated class like this

@Configuration
public class ApplicationConfig {

  @Bean
  public AnyService AnyService() {
      return new AnyServiceImpl();
  }

}

This is equivalent to the following bean definition in standard xml based spring configuration.

<beans>
  <bean id="anyService" class="com.inphina.AnyServiceImpl"/>
</beans> 

Both declarations make a bean called anyService available in the ApplicationContext bound to an object instance of type AnyServiceImpl.

If one bean depended on another bean then defining is as simple as having one bean call the other.

@Configuration
public class ApplicationConfig {
  @Bean
  public DependentBean dependentBean() {
      return new DependentBean(primaryBean());
  }

  @Bean
  public PrimaryBean primaryBean() {
      return new PrimaryBean();
  }
}

It is very easy to take sides here but both the Java based configuration as well as XML based has its own benefits. On one hand XML is declarative, Java is Object Oriented. Most will swear by advantage of one over other. I think both ways provide different semantics of doing the same thing and both approaches to configuration is ok if it suits application needs.

Let’s see how we went about implementing this feature.

Google provides its spreadsheet API and it allows you to create and retrieve data in Google Spreadsheets, but it does not allow us to either create or manage their permissions. Google also has Google Documents List API which can be used to create and retrieve a list of Google Spreadsheets. For our case Google Documents List API provides exporting functionality such that we can export Google documents in common formats like pdf, rtf, xls and others. Google also provides java mail api and with its Multi-Part Messages feature we can send attachment as well.

For implementing this, we used spreadsheet API to retrieve spreadsheet entry. Using the spreadsheet entry we used Google Documents List API to export it in the specified format and we used java mail api to send exported data as an attachment.

We used play framework for our implementation but it can also be applied on other java frameworks as well. The class EmailSender first creates a google spreadsheet service and document service.

public class EmailSender extends Controller {

private static final String META_FEED_URL = "https://spreadsheets.google.com/feeds/spreadsheets/private/full";
private static String spreadSheetName = "spreadsheetname";
private static String yourApplicationName = "emailSender-version1";
private static String userEmail = "yourmail@gmail.com";
private static String recepientEmail = "recepientmail@gmail.com";
private static String password = "password";


public static void emailSpreadsheetAsAttachment() throws AuthenticationException, MessagingException, IOException {
	SpreadsheetService spreadSheetService = createSpreadSheetService();
	DocsService documentService = createDocumentService();
	SpreadsheetEntry spreadsheetEntry = loadSpreadSheet(spreadSheetService, spreadSheetName);
	byte[] spreadSheetData = getSpreadSheetData(spreadSheetService, documentService, spreadsheetEntry);
	sendMailWithAttachment(spreadSheetData);
	}
. . .
. . .
private static SpreadsheetService createSpreadSheetService() throws AuthenticationException {
	SpreadsheetService spreadSheetService = new SpreadsheetService(yourApplicationName);
	spreadSheetService.setUserCredentials(userEmail, password);
	return spreadSheetService;
	}

private static DocsService createDocumentService() throws AuthenticationException {
	DocsService docsService = new DocsService(yourApplicationName);
	docsService.setUserCredentials(userEmail, password);
	return docsService;
       }
. . .
. . .
}

public class EmailSender extends Controller {
. . .

private static SpreadsheetEntry loadSpreadSheet(SpreadsheetService spreadSheetService, String sheetName) {
	SpreadsheetEntry spreadSheetEntry = null;
	try {
		URL metafeedUrl = new URL(META_FEED_URL);
		SpreadsheetFeed feed = spreadSheetService.getFeed(metafeedUrl, SpreadsheetFeed.class);

		List<SpreadsheetEntry> spreadsheets = feed.getEntries();
		for (int i = 0; i < spreadsheets.size(); i++) {
			SpreadsheetEntry entry = spreadsheets.get(i);
			if (entry.getTitle().getPlainText().equals(sheetName)) {
				spreadSheetEntry = entry;
				break;
			}
		}
	} catch (Exception e) {
		Logger.info("Loading Worksheet failed:" + e.getMessage());
	}

	return spreadSheetEntry;
	}

. . .
}

Next we use the Google Documents List API to export the spreadsheet in Excel sheet format. The code listing for method getSpreadSheetData is given below.

public class EmailSender extends Controller {
. . .

private static byte[] getSpreadSheetData(SpreadsheetService spreadSheetService, DocsService documentService,
			SpreadsheetEntry spreadsheetEntry) throws IOException {
	String spreadSheetkey = spreadsheetEntry.getKey();
	byte[] data = null;
	InputStream inStream = null;
	ByteArrayOutputStream outStream = null;

	// Substitute the spreadsheets token for the docs token
	UserToken docsToken = (UserToken) documentService.getAuthTokenFactory().getAuthToken();
	UserToken spreadsheetsToken = (UserToken) spreadSheetService.getAuthTokenFactory().getAuthToken();
	documentService.setUserToken(spreadsheetsToken.getValue());

	String exportUrl = "https://spreadsheets.google.com/feeds/download/spreadsheets" + "/Export?key="
				+ spreadSheetkey + "&fmcmd=" + "xls";

	Logger.info("Exporting document from " + exportUrl);

	MediaContent mc = new MediaContent();
	mc.setUri(exportUrl);

	try {
		MediaSource ms = documentService.getMedia(mc);

		inStream = ms.getInputStream();
		outStream = new ByteArrayOutputStream();

		int c;
		while ((c = inStream.read()) != -1) {
			outStream.write(c);
		}

		// Restore docs token for our DocList client
	} catch (Exception e) {
		e.printStackTrace();
	} finally {
		documentService.setUserToken(docsToken.getValue());
		if (inStream != null) {
			inStream.close();
		}
		if (outStream != null) {
			data = outStream.toByteArray();
			outStream.close();
		}
	}

. . .
}

The important point to note in this method is that we substitute the spreadsheets token for the docs token and change it back to the original in the finally block. Since on Google App Engine we cannot work with the File system therefore we get data in ByteArrayOutputStream instead. You can read more on using the tokens and exporting spreadsheets here. This method returns a byte array of the exported spreadsheet data which we can use to send mail as an attachment.

The next method sendMailWithAttachment(spreadSheetData) uses java mail api Multi-Part Messages to send this spreadSheetData as an attachment. The code listing is given below.

public class EmailSender extends Controller {
. . .

private static void sendMailWithAttachment(byte[] spreadSheetData) throws UnsupportedEncodingException,
			MessagingException {
	Properties props = new Properties();
	Session session = Session.getDefaultInstance(props, null);

	Message message = new MimeMessage(session);
	message.setFrom(new InternetAddress(userEmail, "Administrator"));

	message.addRecipient(Message.RecipientType.TO, new InternetAddress(recepientEmail, recepientEmail));

	String textBody = "Your Excel Sheet as attachment";

	Multipart mp = new MimeMultipart();

	MimeBodyPart textPart = new MimeBodyPart();
	textPart.setContent(textBody, "text/html");
	mp.addBodyPart(textPart);

	MimeBodyPart attachment = new MimeBodyPart();
	String excelFileName = spreadSheetName + ".xls";
	String filename = URLEncoder.encode(excelFileName, "UTF-8");
	attachment.setFileName(filename);
	attachment.setDisposition(Part.ATTACHMENT);

	DataSource src = new ByteArrayDataSource(spreadSheetData, "application/x-ms-excel");
	DataHandler handler = new DataHandler(src);
	attachment.setDataHandler(handler);
	mp.addBodyPart(attachment);

	message.setContent(mp);
	message.setSubject(String.format("Google Spreadsheet as attachment"));

	Transport.send(message);

	Logger.info("Mail with attachment file: " + excelFileName + " sent successfully");

	}

. . .
}

I have uploaded this working project here. You will have to update values for spreadSheetName, yourApplicationName, userEmail, recepientEmail and password accordingly in EmailSender class.

You will also have to download Play-1.0.3.2. Unzip it and add it in the path variable of your operating system.

user@user-laptop:~/emailattachment$ cd <PATH_TO_PROJECT>
user@user-laptop:~/emailattachment$ play eclipsify
user@user-laptop:~/emailattachment$ play run

The play eclipsify command converts the project in eclipse project. Now if you hit the url http://localhost:9000/EmailSender/emailSpreadsheetAsAttachment you should see the email sent successfully in the logs. You will have to deploy this application on Google App Engine to send a mail though. The example shown here mails a google spreadsheet but it will work fine for google documents in general.