Managing Wicket Serialization Problem On Google App Engine

There are several problems of using Wicket on Google App Engine. We are porting a Wicket application on Google App Engine and faced several issues. This post will look at the problems we may encounter while working on a Wicket project on Google App engine and how may we overcome them.

Google App engine will it play suggests that if you have a wicket project you follow this workaround your Wicket project will start working on app engine. Still, one of the error you may encounter while working on app engine are Wicket Serialization errors.

Wicket is very powerful in maintaining application/session state. This enables for example a nice usage of back/forward buttons. In a typical Wicket scenario, this is backed by the DiskPageStore implementation. But on Google App Engine we can’t use the DiskPageStore as it relies on writing to the filesystem. Therefore HttpSessionStore is used in Wicket Application class. In this implementation Wicket components and associated model will be saved in Session and therefore in datastore.

This may result in the session getting bloated till we encounter this error com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large on datastore.

The way to manage this problem is that we use a different pageStore implementation which uses google memcache instead. If memcache implementation is used then the data: that is wicket components and its models will not be associated with the Session and therefore will not end up in datastore.

Memcache also has a limit on the data size limit and you may get an exception if data to be entered in it is high "Caused by: com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call memcache.Set() was too large."

MemcachePageStore that we used has a MAX_PAGES_PER_MAP instance variable and using this we can avoid the "memcache.Set() was too large" errors. Let’s look at how we went about implementing it. We tell our Wicket Application class to use Memcache based page store instead of HttpSessionStore.

public class EhourWebApplication extends AuthenticatedWebApplication {
       . . .
       @Override
	protected ISessionStore newSessionStore() {
		return new SecondLevelCacheSessionStore(this, new MemcachePageStore(3));
		// return new HttpSessionStore(this);
	}
       . . .
}

Managing Cold Start On Google App Engine

One of the major problem working on Google App engine is managing cold start. Well what is it?

Google App engine spins down the application when it is idle. On Google App engine, application does not get a permanent instance of java virtual machine. If there are no activities for a while then the JVM goes cold and application will have to start again to render a request. This is cold start and your application will be sluggish if you render a request to it after a while.

GAE team has accepted the issue and introduced “Ability to reserve instances to reduce application loading overhead” into their roadmap.

This is an issue we traditionally are not aware about while developing an application on Google App engine. If we are working on a greenfield project and we have a liberty of choosing the frameworks then we can stay away from heavy frameworks.

The application we are porting is Spring/Wicket/JPA tehnology stack and we like others were suffering from cold start. When the application warms up all the Spring beans are initialized again. Since we use JPA the EntityManagerFactory adds a significant amount of time for startup.

The quick fix for the cold start till Google App engine team figures it out, is to ping the application every few minutes. This will keep the application from going into the cold storage.

For this implementation to work we require a Servlet which essentially does nothing apart from receiving the request. We also require a cron job which renders request every few minutes. Let’s look at the servlet code listing and its mapping in web.xml.

public class PingServlet extends HttpServlet {

   private Logger logger = Logger.getLogger(PingServlet.class);

   @Override
   protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
	logger.info("Keeping the application warm");
   }

   @Override
   protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
	doGet(req, resp);
   }
}

Generating Excel Files On Google App Engine For A Wicket Application

We are porting an existing Wicket application on Google App Engine. This application uses Wicket, Spring and Hibernate. If you read will it play in app engine, you will notice that Wicket is semi compatible and it does work with workarounds. Hibernate on the other hand is incompatible and Spring is fully supported. We began porting this application after changing persistence to use JPA instead of hibernate. We used workarounds for wicket described here.

This application generates Excel sheets and used Apache POI for generating Excel reports. Apache POI is not supported in Google App engine and it seems that Apache POI will not change their implementation to make it work for Google App Engine in near future.

We started looking at ways to generate Excel sheets. We decided on using java Excel API for our application. Let’s see how can we generate excel files for a Wicket application on Google App engine.

First of all add the maven dependency for JExcel in the application:

<dependency>
    <groupId>net.sourceforge.jexcelapi</groupId>
    <artifactId>jxl</artifactId>
    <version>2.6.10</version>
</dependency>

Problems When Deploying Working Application on Google App Engine

Google Dev and production environment has differences. You may run into problems if you expect that application will also run fine on app engine if there are no errors on app engine development server. We are in process of porting an existing application on Google App engine. The application we are building uses Wicket, Spring and JPA. If you read the will it play in app engine you will find that Spring is compatible, Jpa works with datanucleus implementation and Wicket is semi compatible.

Most of the work required for porting the application required changes in data layer of the application. You can read more about the way to use jpa for persistence in Google App engine here. Wicket does work with the workarounds mentioned here. For wicket application we need to enable sessions in app engine config file.

<sessions-enabled>true</sessions-enabled>

Second we need to disable thread monitoring resource watcher.

@override
protected void init() {
	super.init();
	this.getResourceSettings().setResourcePollFrequency(null);
}

and at last we need to override the newSessionStore() to return HttpSession store because the default second level session store uses java.io.File which Google App Engine does not allow.

@override
protected void newSessionStore() {
	return new HttpSessionStore(this);
}

Generating Charts In Wicket Application On Google App Engine

We are in process of porting an existing Wicket application on Google App engine. This application’s charting engine used Java color classes along with Swing components to generate dynamic images. These images are then used by Wicket to display on the front-end . Unfortunately Google app engine does not support these classes. We therefore had to find an alternative to generate Charts for our application.

We decided upon using google charts in our Wicket based project. We came across this wicket google charts sub project which has the capabilities to generate charts for the application. We found out later that it used awt classes at some parts to generate charts. Looking at the code from this project we realized that it is simple to do this on our own.

Google charts require an URL format which has data along with meta information on how to display it. This well formed URL is embedded in the image tag of the rendered html content is enough to display the chart on the browser. So, in order to generate charts we had to crunch the data in the application to generate a well formed URL. Then we simply had to embed this URL in the image tag to render it on the browser. For example an URL like this:

http://chart.apis.google.com/chart?chs=350x200&chd=s:WORL&cht=bhs&chxt=x,y&chxl=0:|0|5|10|15|20|25|30|35|1:|IBM|GE|TCS|SA

will generate an image like this:

Start Wicket project with maven

Wicket framework is the latest cousin in developing a web based project. Not much documentation is available in exploring it and only a few books are available to be read. It makes more sense to just delve in the code and what best to simply start a project with maven.

Start with executing the following in the command prompt:

user $ mvn archetype:generate

This will provide a host of projects you can start with, maven will ask for the number related to the kind of project you want to start with. You can select wicket-archetype-quickstart which is numbered at 37 for the latest maven.

It will then ask for defaults like artifactId and other values to generate the project structure. If we are using Eclipse IDE, we can generate the classpath by using the following in the command prompt:

user $ mvn eclipse:eclipse

Now import the maven project into eclipse to quick-start with the Wicket application.