Posts Tagged ‘programming’
Tuesday, March 24th, 2009
I’m sure that I’m not the person to make this connection, but it occurred to me the other day that being on a small
team of coders is a lot like being in a band. I’ve been in a couple bands that never went far beyond the garage (I’m allegedly a bass player), so perhaps I’m not the foremost authority on this topic. However, I think there are a few parallels between building, e.g., a small MVC web application, and writing and performing the next standard verse-chorus-verse rock anthem.
In particular, I think there are parallels between the specific members of a prototypical Rock band, and the members of a team who create MVC applications:
- Drummer – The Anchor. Provides the foundation for the music, onto which the other layers are stacked and woven. In an MVC app, this is your database guy/gal. The person modeling your data and managing the schematic underpinnings of your application is your drummer. And if you’re using a weird schema-less database like CouchDB, then you have yourself a sloppy jazz drummer, which means more work for the Bassist.
- Bass Player – The Bass Player sets the groove for the song, and maps the primal beats that the drummer is hammering out into something melodic for the rest of the band to work with. The bassist is also crucial for carrying the beat for the rest of the band when the drummer is screwing around (see note about jazz drummers, above). In our web application, this is the domain layer, where your ORM, caching, and validation all chill out. And like the bass player, this is rarely the sexiest or most glamorous part of the application.
- Guitarist(s) - This is your business logic developer. Some bands choose to break this up into your traditional Angus/Malcom roles of “rhythm” and “lead” guitar. Likewise, you may choose to have separate model and controller layers in your app (particularly if you’re a three-tier app: the web tier controller is your Angus and the middle tier model is your Malcom). Bass players can sometimes get by as guitarists in a pinch, and vice versa. Likewise, you’ll see business logic guys doing domain work and vice versa. Just don’t let them try to play their parts on the wrong instrument – you’ve gotta separate your concerns, dude. Aside from the lyrics, this is the part of the song that your listeners are the most likely to hear.
- Lead Singer – This is your AJAXy, CSS laden, poetic presentation of your band’s message. It’s what your listeners hear (and see) first. And the singer gets all flustered when the rest of the band screws up (the singer is a real primadonna, very sensitive). Changing your lead singer will almost certainly alieniate your fans (think Facebook redesign – the Sammy Hagar of UI decisions).
I tend to get carried away with analogies, and this one is no different – I could keep going (your roadies are your project managers, your label is the marketing and executives who make all the money) but I’ll try to show some restraint. But given this analogy, I find it interesting that the role I play on software teams is often similar to the role I play in a band. I like bass.
Photo Credit: Ian Matthew Soper
Tags: programming
Posted in blog | 2 Comments »
Monday, December 22nd, 2008
Note: This is my first blog post on my newly upgraded Wordpress blog (prior to thi
s, I used Blogger). Hopefully everything goes OK!
Sometimes it’s helpful to do some performance benchmarks on your EJBs. There are a few different ways to do this, but I’ve found that Apache’s JMeter is an excellent tool for benchmarking. Unfortunately, JMeter doesn’t come with a general-purpose sampler for testing arbitrary EJBs. Luckily, it isn’t very difficult to create one.
For this article, I’m using the JBoss application server to host my EJBs. The process for using other containers should be quite similar.
1.) Create a factory to lookup your EJBs.
The first thing that you’ll probably want to do is create a simple singleton factory class to create instances of your EJB client for your test. I use something like the following:
public class MyServiceFactory {
private static final Log log = LogFactory.getLog(MyServiceFactory.class);
private static MyService service;
private static MyServiceFactory me;
private MyServiceFactory() { }
static {
MyServiceFactory.me = new MyServiceFactory();
}
public static MyServiceFactory getInstance() {
return MyServiceFactory.me;
}
public MyService getService() {
if (MyService.service == null) {
// Get the remote interface of the music search service
try {
log.info("Loading the service...");
// JNDI the old-fashioned way:
Context ctx = new InitialContext();
service = (MyService)ctx.lookup("MyAction/remote");
if (service == null) {
log.error("Didn't get the service!");
}
} catch (NamingException e) {
log.error("Error looking up the remote service", e);
return null;
}
}
return service;
}
}2.) Write the test
Next, we’ll need to write the test itself. To do this, we’ll extend the AbstractJavaSamplerClient class in JMeter’s org.apache.jmeter.protocol.java.sampler package. This abstract class has a runTest method that we will override, and this method implements the actual test. We will also override the getDefaultParameters method to provide some reasonable defaults values which will be displayed in JMeter’s GUI application.
package us.mikedesjardins.demo.jmeter;
import org.apache.jmeter.config.Arguments;
import org.apache.jmeter.protocol.java.sampler.AbstractJavaSamplerClient;
import org.apache.jmeter.protocol.java.sampler.JavaSamplerContext;
import org.apache.jmeter.samplers.SampleResult;
public class DigitalContentServiceEJBTestSampler extends AbstractJavaSamplerClient {
public SampleResult runTest(JavaSamplerContext context) {
SampleResult results = new SampleResult();
MyService service = MyServiceFactory.getInstance().getService();
results.sampleStart();
Long param1 = context.getLongParameter("PARAM_1");
String param2 = context.getStringParameter("PARAM_2");
MyResult result = service.myMethod(param1, param2);
if (result != null) {
results.setSuccessful(true);
results.setResponseCodeOK();
results.setResponseMessage("'myResult:" + myResult);
} else {
results.setSuccessful(false);
}
results.sampleEnd();
return results;
}
@Override
public Arguments getDefaultParameters() {
Arguments args = new Arguments();
args.addArgument("PARAM_1", "4815162342");
args.addArgument("PARAM_2", "Iculus");
return args;
}
}3.) Setup JMeter
JMeter’s extra libs directory is ${JMETER_INSTALL_LIB}/lib/ext. Into that directory you will need to copy any jars that your EJB client will need. In you’re using JBoss, you will want to copy the jbossall-client.jar into that directory as well (for the JNDI client and other remoting goodies) – presumably other application servers have similar client jar files available.
When you fire up JMeter, your new sampler should appear in the Samplers menu. Enjoy!
Photo Credit: Bill Jacobus
Tags: ejb, java, jmeter, programming
Posted in blog | 1 Comment »
Monday, September 29th, 2008
At my past few jobs, I’ve had to spend a lot of time on conference calls with developers and project managers. While I don’t profess to be a “conference call expert” (or, for that matter, an etiquette expert), I think there are some general ground rules for conference calls, and I’m going to deviate a bit from my usual Hibernate/JEE topics to itemize the ones that I think are important.
At my current job, I’m actually pretty lucky. Almost all of us work remotely, so we’re pretty accustomed to how to behave in the ubiquitous con-call. At a previous engagement, I wasn’t so lucky – I’ll omit their names to protect the guilty.
So here’s an arbitrary list from me, a random guy on the internet of questionable credentials, of what you should and shouldn’t do on a conference call. Some of these probably seem really obvious, but I’m listing even the obvious ones because some people still break them:
1.) Use the Mute Button, especially on a speakerphone. The most important reason for using Mute is to prevent echo-y feedback, particularly on speaker phones. If you aren’t talking, please put your speaker phone on mute for the sake of everyone else listening. Note that following this rule will invariably lead to the embarrassing situation where you start talking and don’t realize that you’ve left your phone on mute. Don’t worry, we’ve all done it and we’ll all forgive you!
2.) Introduce yourself when you enter the conference. Some conference systems will force you to state your name when you log in, and then the robo-operator will announce you when you join. I actually find this to be quite jarring, especially when robo-operator interrupts what everyone is saying. Better systems will play a quiet tone when people enter or leave the call. Some, like one that a client had at my previous job, will do nothing at all. This would allow people to lurk on the line unbeknownst to the participants, which is terribly rude. Likewise, if you enter a conference room while a conference call is already underway, announce yourself at the next free moment. Ideally, the moderator of the conference will also take a moment to introduce everybody at the beginning of the call if there is a chance that everyone doesn’t know each other. This is a great idea.
3.) Close the Door. It’s best to go to a closed room to join a conference call to eliminate background noise. If you can go to such a place, do it. Don’t join a conference call from, e.g., a street cafe or airport gate. In doing so you risk a passing ambulance, street parade, or TSA threats to destroy unattended luggage interrupting the call.
4.) Don’t use a mobile phone if you can help it. Occasionally you can get away with using a mobile for a con-call in a pinch. But if it’s a super-important meeting where you are trying to make an impression or do something complicated, nothing will be more frustrating than a garbled, broken up, or dropped call. Try to use a land-line if you can.
5.) Don’t eat on the phone. I guess this is just a personal pet peeve of mine. I can’t even listen to NPR if the announcer’s voice is all moist and saliva-sounding. This is more important if you’re wearing a headset.
6.) Have an agenda that everyone receives beforehand. This goes for all meetings, but it’s just as critical for a conference call. This helps you set expectations and stay on-topic during the call.
7.) Get some screen-sharing software. Better yet, get some screen sharing software and test it out before the meeting. With it, you can use something as simple as Notepad or TextMate as a virtual whiteboard. For some reason, we never did this at my previous jobs. We’ve done it at my current job and it works great.
8.) Pick a good time for all participants. This is particularly important if you’re working in different time zones. Generally you can depend on developers being available between 10am and 4pm in their respective time zones – you can argue that programmers need to grow up and work regular business hours like everyone else. But until the voodoo of deadline creation becomes a perfected science, software people will need to occasionally work late into the night or early in the morning to meet schedules. Or (like anyone else) we may have daycare responsibilities and have to leave a little early. So we stay flexible. This means that if you’re working on, e.g., both coasts of the United States, your best bet is between 1pm and 3pm, eastern time. If you’re working in the U.S. and the far east, you’re usually stuck with getting the U.S. participants to work early, and the others late in their workday. Similarly between Europe and the U.S., the U.S. participants will probably need to be available late in the day, and the Europeans earlier.
What do you think? Have I missed any conference call ground rules?
Photo Credit: Stephen Cummings
Tags: industry, programming
Posted in blog | 2 Comments »
Tuesday, August 5th, 2008
One of the ancillary projects of the Hibernate framework is the Hibernate Tools toolset. Using Hibernate Tools, you can automatically generate your mapping files (or, if you prefer, JPA annotations), POJOs, and DDL from your database schema.
I’ve been enamored with the “Convention over Configuration” web frameworks lately (e.g., Grails, Django), and wondered how hard it’d be to reproduce some of their magic ORM functionality using Hibernate Tools. I’ve concluded that I’ve still got a long ways to go, but that I might have some worthwhile tips to share in the meantime.
My progress has been impeded a bit because the documentation for Hibernate Tools is pretty weak. Hibernate Tools was really generated with Eclipse users in mind. A large portion of the documentation is devoted to explaining how to use the IDE. I’m not using Eclipse, and I intend to use the tools at the command-line, and command-line use really isn’t targeted.
0.) Get the tools and dependencies.
I’m still in the dark ages – I use ant instead of maven for builds, and need to track down jar dependencies the old fashioned way. In addition to the usual Hibernate jars, you’ll also need hibernate-tools, freemarker, and jtidy.
1.) Setup an Ant task
This is pretty straightforward, and it is explained fairly well in the documentation. First, define yourself a Hibernate Tools task. Mine looks like this:
<taskdef name="hibernatetool" classname="org.hibernate.tool.ant.HibernateToolTask">
<classpath>
<fileset refid="hibernate.libs" />
<fileset refid="hibernatetools.libs" />
<fileset refid="app.libs" />
<fileset refid="compile.libs" />
<pathelement path="${build}" />
<pathelement path="${basedir}" />
</classpath>
</taskdef>Next, create a task that uses our new definition. We’re going to be creating the mapping files and POJOs in this example. It will look something like this:
<target name="gen_hibernate" depends="compile">
<delete dir="${genhbm}" />
<mkdir dir="${genhbm}" />
<hibernatetool>
<jdbcconfiguration
configurationfile="${src}/hibernate-connection.cfg.xml"
revengfile="hibernate.reveng.xml"
packagename="us.mikedesjardins.data"
detectmanytomany="true"
detectoptimisticlock="true" />
<hbm2hbmxml destdir="${genhbm}" />
<hbm2java destdir="${genhbm}">
<property key="jdk5" value="true" />
<property key="ejb3" value="false" />
</hbm2java>
</hibernatetool>
</target>You’ll note that I’ve created a hibernate configuration file above which only contains connection information. This is because I ran into problems when I tried to use a hibernate configuration with cache configuration elements in it. You’ll also note a reference to a file called hibernate.reveng.xml. This file controls various aspects of the reverse engineering process. Mine is very simple – I’m working with MS SQL Server, and I don’t want it to include any of the system tables which begin with the prefix sys:
<hibernate-reverse-engineering>
<!-- Eliminate system tables -->
<table-filter name="sys.*" exclude="true">
</hibernate-reverse-engineering>
That should be all that you need to do to get started. When you run this ant target, your domain object POJOs and Mappings should end up in the ${genhbm} directory.
2.) First Problem – Table Names
Let’s say you’re in a company that prefixes most of it’s tables with something silly, like “tb_”. In that case, Hibernate tools will happily create all kinds of POJOs for you named TbAccount, TbAddress, etc. This is probably not what you want. Fortunately, Hibernate tools let you provide your own “Reverse Engineering Strategy” class to deal with situations just like this.
First, update your build target in Ant to include a reference to your strategy class as an attribute of the jdbcconfiguration element:
<target name="gen_hibernate" depends="compile">
<delete dir="${genhbm}" />
<mkdir dir="${genhbm}" />
<hibernatetool>
<jdbcconfiguration
configurationfile="${src}/hibernate-connection.cfg.xml"
revengfile="hibernate.reveng.xml"
packagename="us.mikedesjardins.data"
detectmanytomany="true"
detectoptimisticlock="true"
reversestrategy="us.mikedesjardins.hibernate.CustomReverseEngineeringStrategy"/>
<hbm2hbmxml destdir="${genhbm}" />
<hbm2java destdir="${genhbm}">
<property key="jdk5" value="true" />
<property key="ejb3" value="false" />
</hbm2java>
</hibernatetool>
</target>Next, we’ll need to create the strategy class. The documentation recommends that you extend the DelegatingReverseEngineeringStrategy class to do this, like this:
package us.mikedesjardins.hibernate;
import org.hibernate.cfg.reveng.DelegatingReverseEngineeringStrategy;
import org.hibernate.cfg.reveng.ReverseEngineeringStrategy;
public class CustomReverseEngineeringStrategy extends DelegatingReverseEngineeringStrategy {
public CustomReverseEngineeringStrategy(ReverseEngineeringStrategy delegate {
super(delegate);
}
}Unfortunately, I was unable to find any JavaDoc documentation for the Hibernate Tools classes, so I had to do a bit of trial-and-error. It turns out there is a method called tableToClassName that can be overridden. This class accepts a TableIdentifier as its input parameter, and returns the class name in a String. Thus, to remove the tb_ from the class, we could do something like this:
public String tableToClassName(TableIdentifier tableIdentifier) {
String packageName = "us.mikedesjardins.data";
String className = super.tableToClassName(tableIdentifier);
if (className.startsWith("Tb")) {
className = className.substring(2);
}
return className;
}Compile your CustomReverseEngineeringStrategy class, make sure it’s on the hibernatetools task definition’s classpath, and re-run the task. Voila! The Tb’s are gone!
There’s a similar method for column names named columnToPropertyName that handles naming your persisted class’s member variables. In my current project, the primary key columns are named the same as the table name, with “_id” appended to the end. However, in the object model, we like to just name the primary key property “id” to simplify the creation of generic DAOs. I use the columnToPropertyName method for this.
2.) Next Problem – The Generated POJOs need tweaking.
Perhaps the code that Hibernate generates is not to your liking. Maybe you work with standards-compliance-nazis who want a copyright header at the top of each class file. Or maybe all of your persisted objects implement the same interface for use with generic DAOs.
Under the covers, Hibernate uses freemarker to generate POJOs and mapping files. The templates that it uses can be edited to your liking, but doing it is not very straightforward.
First, unzip the hibernate tools jar into a temporary directory. Once unzipped, you’ll find a directory named pojo. This directory contains all of the templates used for POJO generation. Copy that directory into your build area and name it something like hib_templates.
The hbm2java task is actually just an alias for another hibernate tools target called hbmtemplate. With hbmtemplate, you can configure the location of the source templates. So we’ll remove the hbm2java task from the gen_hibernate target, and replace it with an equivalent hbmtemplate task:
<target name="gen_hibernate" depends="compile">
<delete dir="${genhbm}" />
<mkdir dir="${genhbm}" />
<hibernatetool>
<jdbcconfiguration
configurationfile="${src}/hibernate-connection.cfg.xml"
revengfile="hibernate.reveng.xml"
packagename="us.mikedesjardins.data"
detectmanytomany="true"
detectoptimisticlock="true"
reversestrategy="us.mikedesjardins.hibernate.CustomReverseEngineeringStrategy"/>
<hbm2hbmxml destdir="${genhbm}" />
<hbmtemplate templateprefix="pojo/"
destdir="${genhbm}"
template="hib_templates/Pojo.ftl"
filepattern="{package-name}/{class-name}.java">
<property key="jdk5" value="true">
<property key="ejb3" value="false">
</hbmtemplate>
</hibernatetool>
</target>Now that we’ve told hibernate tools where our template lives, we can edit it to our liking. For example, if we want to add a copyright notice to the top of each generated POJO, we could do so thusly:
//
// Copyright 2008 Big Amalgamated Mega Global Software Corp. All Rights Reserved.
//
${pojo.getPackageDeclaration()}
// Generated ${date} by Hibernate Tools ${version}
<#assign classbody>
<#include "PojoTypeDeclaration.ftl"/>; {
.
.
.
(etc)Now that we’ve done this, we can create more templates that, e.g., generate empty DAO classes, automatically create derived classes, etc.
Hope that helps!
Photo Credit: geishaboy500
Tags: hibernate, java, jpa, programming
Posted in blog | 1 Comment »
Saturday, July 12th, 2008
In a recent post, I showed a trick for determining which users in a system are running a long-running query. One commenter suggested that using a full-text search system made a lot of sense in those situations, and I wholeheartedly agree. So I decided to devote a new post to Hibernate Search!
In this example, I provide a live, online interactive search of an online cheese database, and you can download the example (more on that at the end of the post).
Step One – The Test Data
I spent the most time on this project was creating test data for the example program. I headed over to Freebase to see what was available for data sets. Among lots of other things, they have a free database of cheese. First, I downloaded the data in TSV format, dumped it into a raw table, and massaged it into a relational, normalized schema. I ended up with this when I was done:
So, a CHEESE can have only one ORIGIN (a country or region where the cheese is made), is made from one-to-many types of MILK, and may have zero-to-many TEXTURES associated with it.
Step Two – Build the Domain Model
The domain model for this system is very simple. Each Cheese class has an Origin, and a set of Milk and Textures:
@Entity @Table(name="MILK")
public class Milk {
@Id @Column(name="milk_id")
private Integer id;
@Basic @Column(name="name")
private String name;
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}@Entity @Table(name="ORIGIN")
public class Origin {
@Id @Column(name="origin_id")
private Integer id;
@Basic @Column(name="name")
private String name;
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}@Entity @Table(name="TEXTURE")
public class Texture {
@Id @Column(name="texture_id")
private Integer id;
@Basic @Column(name="description")
private String description;
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}@Entity @Table(name="CHEESE")
public class Cheese {
@Id @GeneratedValue(strategy=GenerationType.IDENTITY)
@Column(name="cheese_id",nullable=false,unique=true)
private Integer id;
@Basic @Column(name="name")
private String name;
@ManyToOne(cascade={CascadeType.ALL})
@JoinColumn(name="origin_id",nullable=false)
private Origin origin;
@ManyToMany(cascade={CascadeType.PERSIST,CascadeType.MERGE})
@JoinTable(name="CHEESE_MILK_MAP",
joinColumns=@JoinColumn(name="cheese_id"),
inverseJoinColumns=@JoinColumn(name="milk_id"))
private Set<milk> milks = new HashSet<milk>();
@ManyToMany(cascade={CascadeType.PERSIST,CascadeType.MERGE})
@JoinTable(name="CHEESE_TEXTURE_MAP",
joinColumns=@JoinColumn(name="cheese_id"),
inverseJoinColumns=@JoinColumn(name="texture_id"))
private Set<texture> textures = new HashSet<texture>();
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}Step Three – Add Search Annotations and Configure Lucene
Behind the scenes, Hibernate Search uses the Apache Lucene search engine to do its indexing. In short, it maintains a mapping of object IDs to search terms in an external file, and updates the file when objects are added, updated, or deleted. To start using Hibernate Search, you’ll need to configure the location of these index files, as well as a search directory provider (we’ll just use the default). This is done in your Hibernate properties file, or (if you use JPA, like me), in persistence.xml:
<property name="hibernate.search.default.directory_provider" value="org.hibernate.search.store.FSDirectoryProvider" />
<property name="hibernate.search.default.indexBase" value="/var/lucene/cheese-indexes" />
Next, you need to indicate to Lucene which classes need to be indexed. You also need to indicate which data fields 1.) contain the document ID, and 2.) contain relevant search text. In our example, we only need to index the Cheese objects. We want to allow users to search on cheese name, milk name, origin, and texture.
First, we indicate that we want to index the Cheese objects by applying the @Indexed annotation to the class, and we elect to use the Id field to identify the Cheese objects to Lucene by applying a @DocumentId annotation to it. Next, we indicate that the cheese name contains searchable text by adding the @Field(index=Index.TOKENIZED, store=Store.NO) annotation to it. The annotation parameters are informing Hibernate Search to use Lucene’s default tokenizer to summarize the text, and not to store a copy of the document content.
We also want to allow users to search on Origin, Milk Name, and Texture. This text is not contained within the Cheese object, instead they’re in related object. So we need to add the @IndexEmbedded annotation to the member variables in the Cheese class that refer to the objects which contain the searchable text, and we also need to add the @Field(index=Index.TOKENIZED, store=Store.NO) annotation to the Milk, Origin, and Texture classes to indicate which fields are searchable.
When you’re done, the modified domain classes will look like this:
@Entity @Table(name="MILK")
public class Milk {
@Id @Column(name="milk_id")
private Integer id;
@Field(index=Index.TOKENIZED)
@Basic @Column(name="name")
private String name;
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}@Entity @Table(name="ORIGIN")
public class Origin {
@Id @Column(name="origin_id")
private Integer id;
@Field(index=Index.TOKENIZED)
@Basic @Column(name="name")
private String name;
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}@Entity @Table(name="TEXTURE")
public class Texture {
@Id @Column(name="texture_id")
private Integer id;
@Field(index=Index.TOKENIZED)
@Basic @Column(name="description")
private String description;
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}@Indexed
@Entity @Table(name="CHEESE")
public class Cheese {
@DocumentId
@Id @GeneratedValue(strategy=GenerationType.IDENTITY)
@Column(name="cheese_id",nullable=false,unique=true)
private Integer id;
@Basic @Column(name="name")
@Field(index=Index.TOKENIZED, store=Store.NO)
private String name;
@IndexedEmbedded
@ManyToOne(cascade={CascadeType.ALL})
@JoinColumn(name="origin_id",nullable=false)
private Origin origin;
@IndexedEmbedded
@ManyToMany(cascade={CascadeType.PERSIST,CascadeType.MERGE})
@JoinTable(name="CHEESE_MILK_MAP",
joinColumns=@JoinColumn(name="cheese_id"),
inverseJoinColumns=@JoinColumn(name="milk_id"))
private Set<milk> milks = new HashSet<milk>();
@IndexedEmbedded
@ManyToMany(cascade={CascadeType.PERSIST,CascadeType.MERGE})
@JoinTable(name="CHEESE_TEXTURE_MAP",
joinColumns=@JoinColumn(name="cheese_id"),
inverseJoinColumns=@JoinColumn(name="texture_id"))
private Set<texture> textures = new HashSet<texture>();
@Version @Column(name="version")
private Integer version;
// Accessors omitted
}Step Four – The Servlets
In this example, I didn’t want to rely on any web frameworks or even on JSPs, so I wrote a good old-fashioned servlet to exercise the search function. No sane person would ever do it this way. There are actually two servlets in the example – one for application initialization and one for the page itself.
The initialization servlet does the work of indexing all of the database data the first time through. For our small data set, this takes less than a minute. For larger data sets, it wouldn’t make sense to re-index everything every time you start the application. The initialization code iterates over all of the Cheese objects ant tells the FullTextEntityManger to index it:
public void init() {
Dao<cheese> dao = new CheeseDao();
EntityManager em = dao.getEntityManager();
FullTextEntityManager fullTextEntityManager = Search.createFullTextEntityManager(em);
List<cheese> cheeses = em.createQuery("select c from Cheese as c").getResultList();
for (Cheese cheese : cheeses) {
fullTextEntityManager.index(cheese);
}
}The main page servlet has some code in the doPost method to perform the search based on the contents of the text form field. That code looks like this (I’ve shortened it up a bit here by removing some error checking and HTML output):
public void doPost(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
response.setContentType("text/html");
PrintWriter out = response.getWriter();
emitHeader(out);
String searchTerm = request.getParameter("searchterm");
EntityManager em = dao.getEntityManager();
FullTextEntityManager fullTextEntityManager =
org.hibernate.search.jpa.Search.createFullTextEntityManager(em);
MultiFieldQueryParser parser =
new MultiFieldQueryParser( new String[]{"name",
"origin.name",
"milks.name",
"textures.description"},
new StandardAnalyzer());
try {
org.apache.lucene.search.Query query = parser.parse(searchTerm);
javax.persistence.Query hibQuery =
fullTextEntityManager.createFullTextQuery(query,Cheese.class);
List<cheese> result = hibQuery.getResultList();
emitTable(out,result);
} catch (ParseException e) {
log.error("Got a parse exception", e);
throw new ServletException(e.getMessage());
}
emitFooter(out);
out.close();
}That’s all there is to it! As you can see, setting up Hibernate Search is very simple. Most of the effort for this project was spent creating the data and making the servlets work.
Enjoy the Finished Product
To see this less-than-world-changing application in action, visit it here. You can also download the whole eclipse project and try it out for yourself. It comes with SQL dumps suitable for MySQL and PostgreSQL, and it has been tested with both environments under Tomcat 5.5.
Photo Credit: Chris Buecheler
Tags: hibernate, java, jpa, programming
Posted in blog | 1 Comment »
Monday, June 23rd, 2008
Your Problem
You have a data model with table that contains data you want to aggregate. For instance (returning to my venerable Pizza Shop example), let’s say you have a PRODUCT table that enumerates the items your pizza shops sells, and a LOCATION table that contains all of your retail locations:
We also have a table that contains the sum of all of the previous days’ sales, broken down by PRODUCT and LOCATION:
This is all well and good, but we’ve been asked to create an Executive Dashboard for the President of the pizza chain, and she would like to see daily sales by product. She is not interested in a breakdown by location.
We could tally it up client side…
What if we just loaded the entire table into the client, and iterated over the per-product results, and present that? Unfortunately, ORM libraries are pretty stupid in situations like these, and will generate all kinds of expensive reads to the database when you try to solve the problem this way. If you want to learn more about lazy loading, and why you shouldn’t iterate over a collection, check out my earlier post here.
We could just create a view in the Database…
We could just create a view, and aggregate the data there. Then we can easily create a Hibernate mapping to that view. The query for the view is simple:
CREATE VIEW sales_by_product_view AS
SELECT product_id, sum(total_sales) AS total_sales
FROM YESTERDAY_SALES
GROUP BY product_id
However, we are working with a tyrannical DBA. She is not keen on proliferating views throughout our otherwise pristine schema every time the President has decided that the company needs a new widget for the executive dashboard application.
…or we could fake it with a custom loader
Instead, let’s create a Hibernate mapping that generates the same results as the view. First, let’s create a simple POJO to contain the results:
package us.mikedesjardins;
public class SalesByProduct {
private Integer id;
private Integer productId;
private BigDecimal sales;
// accessors omitted
}The corresponding mapping file would look like this. I re-used the product_id for the ID in this example. Note the loader element:
<hibernate-mapping package="us.mikedesjardins">
<class name="SalesByProduct"
dynamic-insert="false"
dynamic-update="false"
mutable="false">
<id name="id" type="int" unsaved-value="null">
<column name="__id" sql-type="int identity" not-null="true" unique="true" />
</id>
<property name="productId" type="int">
<column name="__product_id" not-null="false" />
</property>
<property name="sales" type="java.math.BigDecimal">
<column name="__total_sales" not-null="false" />
</property>
<loader query-ref="salesByProductQuery" />
</class>
<sql-query name="salesByProductQuery">
<return class="SalesByProduct" />
<![CDATA[
select product_id as __id
, product_id as __product_id
, sum(total_sales) as __total_sales
from YESTERDAY_SALES
where product_id = :product_id
group by product_id
]]>
</sql-query>
</hibernate-mapping>Note that you need to put a positional argument in the query, or Hibernate will get nasty about parsing it. Hope that helps!
Tags: hibernate, java, jpa, programming
Posted in blog | 1 Comment »
Monday, June 16th, 2008
Yesterday I read a blog post by Kenneth Downs entitled “Why I Do Not Use ORM” on his The Database Programmer blog. It wasn’t the first blog post with gripes about Object Relational Mapping, and it certainly won’t be the last. For me, this particular article highlighted a few misconceptions about why and how ORM should be used, and I thought I might chime in with my own perspective.
ORM is not a way to avoid SQL
The first thing that stood out for me was this quote:
“The language SQL is the most widely supported, implemented, and used way to connect to databases. But since most of us have long lists of complaints about the language, we end up writing abstraction layers that make it easier for us to avoid coding SQL directly.”
To his credit, the author wasn’t actually addressing ORM in this paragraph. However, anyone writing business logic which interacts with a database, who is unable to write some basic SQL, is like a bull in a china shop; nothing good will come of it regardless of the tools and abstraction layers employed.
The Simple Example
The next example in the post shows how one would write a row to a database in only four lines of PHP code – it looks something like this (I removed the comments):
$row = getPostStartingWith("inp_");
$table_id = myGetPostVarFunction('table_id');
if (!SQLX_Insert($table_id,$row)) {
myFrameworkErrorReporting();
}I don’t know PHP, but I think the code is reading every field in a posted form that starts with inp_, generates an insert statement straight from the ID’s on the form fields, and writes an insert statement with the results.
Perhaps it’s unfair to criticize this code because “it’s just a simple example,” but if this code is being held up as an example of how short and simple non-ORM code can be, one does have to wonder
- When the database schema changes, does the HTML need to be updated so that the form fields match the database schema?
- Where are the transactions? What if I need to insert into several tables and roll back the transaction if one fails?
- Does the handy SQLX_insert method prevent SQL injection attacks?
He goes on to say that this task is made even easier by using a data dictionary to generate the SQL. After reading the “Using a Data Dictionary” article, one has to wonder whether or not the author realizes that it is a very crude form of ORM.
What about Business Logic?
Kenneth Downs tries to head-off any arguments about business logic before they come up, knowing that ORM evangelists will argue that the domain objects can encapsulate essential business logic for the application. His response?
“The SQLX_Insert() routine can call out to functions (fast) or objects (much slower) that massage data before and after the insert operation. I will be demonstrating some of these techniques in future essays, but of course the best permforming and safest method is to use triggers.”
For me, this sounds alarm bells. Triggers slow down transactions. Triggers are in your database, which is often your system’s biggest bottleneck. Triggers silently do things behind your back without telling you. Triggers change databases from vast, efficient places to store relational data, into a lumbering behemoth interpreting procedural code inside big iron.
Conversely, business logic that can be easily distributed across many smaller web servers scales horizontally. The domain layer is a fantastic place to embed simple data massaging – sadly, I often see a pile of persistent entities with getters and setters that don’t do anything.
Counter Example: The Disaster Scenario
Lastly, Ken (can I call him that? What is the ettiquite for this sort of thing, anyway?) shows an example of a piece of code that is likely to cause hundreds of unneeded reads to the database in an untuned ORM-based system. I don’t dispute this; in fact, I posted about an almost identical nightmare scenario myself a while ago.
For this, I go back to my “bull in a china shop” analogy. Programmers can write horrible code in any language, with any tool. Layers of abstraction are a double-edged sword, because you need to understand what they are doing for you. But it’s not the tool’s fault; it’s the person misusing it.
Computer Science’s Vietnam
In 2004, Ted Neward famously called ORM Computer Science’s Vietnam. Encouraged by early successes, we got sucked into the quagmire. There are plenty of reasons to be frustrated with ORM, but I’m not sure I agree with Ken’s. I try to hit the 80/20 rule with ORM, and use it where it makes sense. When I get into a convoluted transaction or need to do a large batch of operations, I’m not afraid to dive into SQL and do the work in a stored procedure. I think it’s a good mix. How about you?
Photo Credit: Ryan Dickey
Tags: hibernate, jpa, programming
Posted in blog | 12 Comments »
Friday, June 6th, 2008
(to the tune of The Spiderman Theme Song from the original cartoon series)
Hibernate, Hibernate
O/R Mapping sure is great
Gavin King, and H-Q-L
Make my life a living hell
Look out! Here comes Hibernate!
Mapping Files in Hibernate
Wish my team would annotate
Are its queries optimized?
Who knows – it’s always a surprise
Hey there, there goes Hibernate!
Don’t fetch data now
Lazy load, ‘cuz you might guess
I don’t need it now
Hibernate, you do know best!
Hibernate, Hibernate
Friendly neighborhood Hibernate
Grab the jars, and climb aboard
OO models are your reward
To it…
Relational DBs are old-school
When you need a complex tool
You need the Hibernate!
Tags: hibernate, java, jpa, programming
Posted in blog | 1 Comment »
Monday, February 18th, 2008
One feature of JPA that didn’t exist in plain-vanilla Hibernate is support for Enumerated types. I haven’t seen a lot of examples of this in practice or on the internet, so in this post I’ll show one example of how to use JDK 5 enumerations with JPA.
For our example, we are going to create an inventory system for our sock drawer. It is comprised of only two tables. The first table, called SOCK, contains one row per sock in our drawer. The columns of the table are:
- sock_id – an auto-incrementing identity column.
- sock_description – a varchar column for a free-form text description of the sock.
- sock_pattern_id – a reference to a row in the SOCK_PATTERN table.
As you may have guessed, the SOCK_PATTERN table looks like this:
- sock_pattern_id – an integer primary key. It’s not auto-incrementing, because we will want to have control over the contents of the field.
- sock_pattern_description – a varchar column for a free-form text description of the pattern.
We need to “prime” our SOCK_PATTERN table with the valid patterns and create a foreign key relationship between the two tables:
INSERT INTO SOCK_PATTERN (sock_pattern_id,sock_pattern_description) VALUES (0,'SOLID');
INSERT INTO SOCK_PATTERN (sock_pattern_id,sock_pattern_description) VALUES (1,'STRIPES');
INSERT INTO SOCK_PATTERN (sock_pattern_id,sock_pattern_description) VALUES (2,'POLKA_DOT');
INSERT INTO SOCK_PATTERN (sock_pattern_id,sock_pattern_description) VALUES (3,'ARGYLE');
Note that we populated sock_pattern_id starting with zero; this is important because the Enumeration below is zero-indexed.
Next, let’s create the classes:
public enum SockPattern {
SOLID, STRIPES, POLKA_DOT, ARGYLE
}
@Entity @Table(name="SOCK")
public class Sock {
@Id @GeneratedValue(strategy=GenerationType.IDENTITY)
@Column(name="sock_id")
private Integer id;
@Column(name="sock_description")
private String description;
@Enumerated @Column(name="sock_pattern_id")
private SockPattern pattern;
public Integer getId() { return id; }
public void setId(Integer id) { this.id = id; }
public String getDescription() { return description; }
public void setDescription(String description) { this.description = description; }
public SockPattern getPattern() { return pattern; }
public void setPattern(SockPattern pattern) { this.pattern = pattern; }
}Now, to use our wonderful contraption, you’d do something like the following:
public class Demo {
private static EntityManagerFactory emf;
static {
Demo.emf = Persistence.createEntityManagerFactory("sockdrawer");
}
@Test
public void socksOne() {
Sock sock = new Sock();
sock.setDescription("My favorite sock.");
sock.setPattern(SockPattern.ARGYLE);
EntityManager em = Demo.emf.createEntityManager();
em.getTransaction().begin();
em.persist(sock);
em.getTransaction().commit();
em.close();
}
}There are some obvious pitfalls to this approach. The object/relational mapping is very brittle; for this code to work, the IDs in the database always need to match the values that the ORM tool gets from the enumeration. Changing these values at a later date could cause some surprising results, and you can’t insert new rows without updating the Enumeration and recompiling. It does obviate the need to map the SOCK_PATTERN table, and you won’t need to worry about the details of cascading the persistent state of related Sock and SockPattern objects.
It’s just a new tool in the JPA toolbox.
(The code in this blog post was tested w/ Postgres and Toplink)
Tags: hibernate, java, jpa, programming
Posted in blog | 1 Comment »
Thursday, January 31st, 2008
So, another JPA tutorial. What makes this one different? Well, for one thing, this one comes with a working, downloadable project that works with Eclipse, NetBeans, and IntelliJ IDEA 7. It’s packaged with Hibernate, Toplink, and OpenJPA. And it’s been tested with MySQL, PostgreSQL, MS SQL Server, and Sybase. In other words, it works with 36 different IDE/JPA Provider/Database combinations!
Another thing that makes this tutorial different is the subject matter: Pizza! Who doesn’t love pizza? Except lactose intolerant people. And people who can’t eat gluten. But other than them, who doesn’t love pizza? So we’re going to create a simple database model for a pizza shop’s point-of-sale system.
The Schema
Unsurprisingly, the starting point for any ORM task is usually the database schema (there are people who start with the Objects and work “backward” to the schema, but I haven’t worked with any of them yet). In our example, we have a pristine, consistent, completely normalized schema. In other words, it’s probably nothing like you’ll ever be lucky enough to see in the real world! Here’s our simple little ERD:
From this ERD, we can infer the following: 1.) An order is comprised of zero or more pizzas. 2.) A pizza is associated with one size. 3.) A pizza may be associated with a string of text containing “special instructions.” 4.) A pizza may have zero or more toppings. You probably also notice that each table has a column called version. This will be used for an optimistic locking strategy.
The first question is, “Where should we start?” There’s no right answer for this, but I find that it’s easiest to start working with the entities with the fewest dependencies. For example, you can’t have an order without a pizza, and you can’t have a pizza without a size, so maybe it makes sense to start with the size. But before that, we’ll want an ID interface.
An ID Interface
First things first. You’ll notice that, in our schema, every table has an integer ID. It’s often a good idea to have all of your objects implement the same interface for accessing the ID, because it makes it easier to create Generic DAOs (more about that in a future post). For now, let’s make a really simple interface like this:
public interface IdObject {
public void setId(Integer id);
public Integer getId();
}Many-to-One Unidirectional Relationships
Now that we’ve gotten that out of the way, let’s create the Size class. It’s a simple POJO littered with annotations, like this:
@Entity
@Table(name="PIZZA_SIZE")
public class Size implements IdObject {
@Id
@Column(name="pizza_size_id")
private Integer id;
@Column(name="pizza_size_description")
private String description;
@Column(name="pizza_size_base_price")
private BigDecimal basePrice;
@Version @Column(name="version")
private Integer version;
public Integer getId() { return id; }
public void setId(Integer id) { this.id = id; }
public String getDescription() { return description; }
public void setDescription(String description) { this.description = description; }
public BigDecimal getBasePrice() { return basePrice; }
public void setBasePrice(BigDecimal basePrice) { this.basePrice = basePrice; }
public Integer getVersion() { return version; }
public void setVersion(Integer version) { this.version = version; }
}Here are some notes on the annotations that were used in the above class:
@Entity tells the JPA provider that this is a managed object.
@Table specifies the table name. The JPA provider will attempt to default the table name to a sane value based on the class name, but I like to be explicit. I’m funny that way. Perhaps it’s OCD. Or a power-trip.
@Column indicates the name of the column, and can include other attributes about the column (you’ll see a few additional attributes later on). Again, JPA can try to default this to sane values for you, but I like to be explicit.
@Version indicates that a particular column is used for indicating when a row is updated. This column can then be used in an optimistic locking scheme.
Next, let’s do the Pizza object to show how we map a Pizza to a Size.
One thing I that always tripped me up when I started out with ORM
tools was the difference between “One-to-Many” and “Many-to-One.” I never knew, if I call a relationship many-to-one in my metadata, is this object the one that there are many of, or is it the other way around? The answer is that “this” object always comes first. ManyToOne means that “there are many of this object to one of those objects.” The “Many” side is often the side that has the foreign key.
In our case, there will be many Pizzas that are the same size. So when we make our Pizza object, we will want to use the @ManyToOne annotation. Here’s what the Pizza object looks like so far. I’ve omitted the getters and setters to save space:
@Entity
@Table(name="PIZZA")
public class Pizza implements IdObject {
@Id
@Column(name="pizza_id")
private Integer id;
@ManyToOne(cascade={CascadeType.ALL})
@JoinColumn(name="pizza_size_id",nullable=false)
private Size size;
@Version @Column(name="version")
private Integer version;
// (Accessor methods omitted)
}Note that, when we made our Size object, we did not include a reference to the Pizza. That was an intentional design decision. In this application, it’s unlikely that we will want to instantiate a Size object, and get a collection containing all of the Pizzas of that size, so we don’t bother with mapping it. This is called unidirectional association.
The @ManyToOne annotation specifies a cascade attribute. There are several different settings for this attribute, which you can read more about here. I tend to cascade the persistent state to all related objects because it reduces the amount of redundant API calls. By default, JPA does not cascade pers istence to related objects. I’ll cover the cascade attribute in future posts, but for now, we’ll go with my personal preference, because I’m writing the article!
The @JoinColumn annotation indicates the column name that defines the linkage between the Pizza and the Size. You’ll also note that we’ve included some additional attributes on our @Column and @JoinColumn annotations. The unique and nullable attributes are particularly useful if you use tools to generate schema DDL from your mappings.
One-to-Many bidirectional relationships
Both the SpecialInstruction and the Order objects are examples of One-to-Many bidirectional relationships. In the case of SpecialInstruction, it is likely that we will care about which Pizza an instruction is associated with, and likewise for the Order. A bidirectional one-to-many relationship implies that one object has a collection of other o bjects. For example, an Order has a collection of Pizzas. First, lets add an order attribute to our Pizza object:
@Entity
@Table(name="PIZZA")
public class Pizza implements IdObject {
.
.
@ManyToOne(cascade={CascadeType.ALL})
@JoinColumn(name="pizza_order_id",nullable=false)
private Order order;
public Order getOrder() { return order; }
public void setOrder(Order order) { this.order = order; }
.
.
}Next, let’s create an Order object to contain our collection of Pizzas, like this:
@Entity
@Table(name="PIZZA_ORDER")
public class Order implements IdObject {
@Id
@Column(name="pizza_order_id")
private Integer id;
@OneToMany(cascade={CascadeType.ALL},mappedBy="order")
private Set pizzas = new HashSet();
@Version @Column(name="version")
private Integer version;
// (version and id accessors omitted)
public Set getPizzas() { return pizzas; }
public void setPizzas(Set pizzas) { this.pizzas = pizzas; }
public void addPizza(Pizza pizza) { pizza.setOrder(this); this.pizzas.add(pizza); }
}There are a couple of things you worth noting about this mapping:
1.) The @OneToMany annotation uses the mappedBy attribute to indicate which member of the related object defines the linkage between the two tables. In this case, we are saying that the Pizza object contains a member named order, which defines the linkage between the two objects.
2.) I’ve created a utility method called addPizza. This simplifies setting both sides of the bidirectional relationship by setting the Order object on the Pizza and adding the Pizza to the Order’s collection. Users of this class will only need to make one method call to do both.
Many-to-Many relationships via a Join Table
The last thing we’ll cover is how to map Join Tables. In our ERD, you can see that Toppings are modeled in the database with a TOPPING table that contains all of the valid toppings, a PIZZA table that contains all of the valid Pizzas, and a PIZZA_TOPPING table in the middle that maps all of the valid Pizzas to all of the valid Toppings. You could create an object called PizzaTopping that corresponds to the PIZZA_TOPPING table. Then you could have a One-to-Many relationship from the Pizza to the PizzaTopping, and a One-to-One from each PizzaTopping to a Topping. That would be very cumbersome to work with! Fortunately, there’s a better way.
Logically, a Pizza has a collection of Toppings. In our Java code, we really shouldn’t care about the fact that there is a PIZZA_TOPPING join table in the middle. First, let’s create a simple Topping class:
@Entity
@Table(name="TOPPING")
public class Topping implements IdObject {
@Id
@Column(name="topping_id")
private Integer id;
@Column(name="topping_description")
private String description;
@Column(name="topping_price")
private BigDecimal price;
@Version @Column(name="version")
private Integer version;
// (Accessors omitted)
}This is how the association would be mapped in the Pizza class:
@Entity
@Table(name="PIZZA")
public class Pizza implements IdObject {
.
.
@ManyToMany(cascade={CascadeType.ALL})
@JoinTable(name="PIZZA_TOPPING",
joinColumns=@JoinColumn(name="pizza_id"),
inverseJoinColumns=@JoinColumn(name="topping_id"))
private Set toppings = new HashSet();
public Set getToppings() { return toppings; }
public void setToppings(Set toppings) { this.toppings = toppings; }
public void addTopping(Topping topping) { this.toppings.add(topping); }
.
.
}The @JoinTable annotation defines three key attributes:
name: Identifies the name of the join table.
joinColumns: This attribute identifies the column name in the join table that points to this object.
inverseJoinColumns: This attribute defines the column name in the join table that points to the other objects.
From your Java code, the semantics for dealing with toppings on a pizza are just like any other set, much like you’d work with a One-to-many object.
Conclusion
Since the introduction of annotations, object/relational mapping is one of the easiest aspects of working with JPA over other frameworks. You can see this whole project in action on your IDE of choice by downloading it here (or from here if that doesn’t work for some reason). It should be a pretty reasonable starting point if you want a reference project to start playing with JPA. I hope to revisit the Pizza Shop project to cover other JPA topics in future posts.
Tags: hibernate, java, jpa, pizzashop, programming
Posted in blog | 8 Comments »
