Chapter 9. DAO

This chapter explores the design of GWT Dao. Data Access Object (DAO) layer allows the app flip between ORM effortlessly. In the last chapter, we introduced three ORM, and now they help us to verify the design of GWT DAO layer.

DAO design

Service layer of the application interacts with the database through DAO layer. To accomodate multiple ORM, DAO layer consists of an Abstract Factory and an Interface to access data.

DAO layer is split into following packages

  • abstract classes and interfaces are in package in.fins.server.dao.

  • implementation are in ORM wise packages in.fins.server.dao.jdo, in.fins.server.dao.hibernate and in.fins.server.dao.mybatis,

Following figure shows the SymbolService interactions with DAO layer.

DAO design
Figure 9.1. DAO design
  1. SymbolService requests abstract factory, in.fins.server.dao.DaoFactory, for an instance of DaoFactory specific to an ORM.

    • When ORM is set to JDO, abstract factory in.fins.server.dao.DaoFactory returns JDO DaoFactory which is in.fins.server.dao.jdo.DaoFactory. Please note that abstract DaoFactory is in in.fins.server.dao package, and JDO DaoFactory which extends abstract DaoFactory is in in.fins.server.dao.jdo package.
  2. Interaction between service layer and data layer happens through an Interface IDao<T>. ORM specific Dao, like in.fins.server.dao.jdo.Dao<T> and in.fins.server.dao.hibernate.Dao<T> etc., implements the IDao interface. These Dao classes use ORM specific constructs to interact with the database.

    • When ORM is set to JDO, SymbolService gets the JDO DaoFactory, it calls DaoFactory.getDao<T>() method which returns in.fins.server.dao.jdo.Dao<T> which implements IDao<T> and uses JDO to interact with database.

    • When ORM is set to Hibernate, then Step 2 returns in.fins.server.dao.hibernate.DaoFactory, and in Step 3, when SymbolService calls its getDao() method, it returns in.fins.server.dao.hibernate.Dao<T> which uses Hibernate to interact with the database.

  3. Based on Type <T>, method DaoFactory.getDao() returns following variants of Dao ( when ORM is set to JDO )

    • getDao<Symbol>() returns in.fins.server.dao.jdo.SymbolDao.

    • getDao<DataGroup>() returns in.fins.server.dao.jdo.DataGroupDao.

    • getDao<Data>() returns in.fins.server.dao.jdo.DataDao.

    • for any other types, getDao() returns in.fins.server.dao.jdo.Dao which is generic Dao capable of handling any type.

    They all implements IDao interface. Even though generic Dao is able to handle any type, including Symbol,DataGroup and Data types, we require type specific Dao as we do some post processing for Symbol, DataGroup and Data.

  4. SymbolService call Dao<T> methods to access database.

DataNucleus Plugin

In the last chapter, we enhanced persistence classes through an Ant target. DataNucleus has a plugin for this. Install the plugin using update site https://www.datanucleus.org/downloads/eclipse-update, before running the example code.

To enable DataNucleus support, invoke project context menu and select DataNucleusAdd DataNucleus Support . To run enhancer, invoke project context menu and select DataNucleus and choose Run Enhancer Tool or Enable Auto-Enhancement

When persistence classes are not enhanced, Dao throws NucleusUserException.

Let’s explore the actual implementation of the design by taking JDO as ORM.

ORM selection

ORM type is set in the application configuration file META-INF/fins.properties.

src/META-INF/fins.properties

# allowed values JDO|MyBatis|Hibernate

orm=JDO

Fins uses in.fins.server.listener/FinsContextListener, which implements ServletContextListener, to read the property file at application startup and set the ORM name as a global attribute in ServletContext. SymbolService, which is also a servlet, retrieves ORM name through ServletContext.getAttribute() method and initializes the appropriate appropriate DaoFactory. Listener entry in web.xml enables FinsContextListener.

 
 
Configuration files

ORM configuration files are placed in src/META-INF directory which is standard place for web application conf files. Hibernate and MyBatis configuration files are same ones from RStore.

But for JDO, instead of datanucleus.properties, we use jdoconfig.xml which is xml version of property file which defines persistence-manager-factory named “datastore”. This feature, known as Named PMF, is the preferred way to configure JDO for web applications. JDOHelper.getPersistenceManagerFactory() constructs the factory by using properties set in jdoconfig.xml file.

in.fins.server.dao.jdo/PMF.java

public final class PMF {
        private static final PersistenceManagerFactory pmfInstance = JDOHelper
                        .getPersistenceManagerFactory("datastore");

        public static PersistenceManagerFactory get() {
                return pmfInstance;
        }
}
Mapping files

Mapping files are placed along with domain class in package in.fins.shared and are the same as ones from RStore.

DAO Layer

Abstract classes and interfaces are in package in.fins.server.dao. Service layer interact through the classes and interfaces of this package to ensure portability between various. ORM. This package contains three class/interface.

  • DaoFactory - is an abstract class with two methods, getDaoFactory() and getDao(). The method getDaoFactory() creates and returns appropriate DaoFactory based on ORM type.

    in.fins.server.dao/DaoFactory.java

            public static DaoFactory getDaoFactory(ORM orm) throws Exception {
                    if (INSTANCE == null) {
                            switch (orm) {
                            case JDO:
                                    INSTANCE = new in.fins.server.dao.jdo.DaoFactory();
                                    break;
                            case MyBatis:
                                    INSTANCE = new in.fins.server.dao.mybatis.DaoFactory();
                                    break;
                            case Hibernate:
                                    INSTANCE = new
                                        in.fins.server.dao.hibernate.DaoFactory();
                                    break;
                            }
                    }
                    return INSTANCE;
            }
    

    DaoFactory.getDao() is an abstract method and it’s implementation is delegated to subclasses like in.fins.server.dao.jdo.DaoFactory.

  • IDao<T> - service layer uses IDao<T> interface, which defines two methods, to interact with DAO layer.

    • selectById() - selects object like Symbol, DataGroup and Data by its ID (primary key)

    • select() - selects using some statement or query, for example, to select a list of symbol names.

    Generally, this interface also defines methods like create, insert, delete and other varieties of select etc., but at this juncture these two methods are sufficient for Fins, and we add insert method in a later chapter. Readers may add and implement other methods as an exercise.

  • DaoHelper - Dao uses this utility class which has methods, to filter the object contents based on FilterMap sent by the client. We will explain this concept shortly.

JDO DAO

JDO specific DAO implementation are in in.fins.server.dao.jdo package. Hibernate and MyBatis are in in.fins.server.dao.hibernate and in.fins.server.dao.mybatis respectively and their classes are similar to JDO except that they make use Hibernate and MyBatis constructs.

Package in.fins.server.dao.jdo contains following classes.

DaoFactory - extends abstract DaoFactory and implements the abstract method getDao()

in.fins.server.dao/DaoFactory.java

        private PersistenceManagerFactory pmf;

        public DaoFactory() {
                pmf = PMF.get();
        }

        @Override
        public <T> IDao<T> getDao(Class<T> clz) throws Exception {
                if (clz == Symbol.class) {
                        log.fine("JDO SymbolDao returned");
                        return (IDao<T>) new SymbolDao<T>(pmf);
                }
                if (clz == Data.class) {
                        log.fine("JDO DataDao returned");
                        return (IDao<T>) new DataDao<T>(pmf);
                }
                if (clz == DataGroup.class) {
                        log.fine("JDO DataGroupDao returned");
                        return (IDao<T>) new DataGroupDao<T>(pmf);
                }
                log.fine("JDO Generic Dao returned");
                return (IDao<T>) new Dao<T>(pmf);
        }

JDO DaoFactory constructor creates JDO PersistenceManagerFactory and assigns it to a field. Method getDao() returns type specific Dao or generic Dao depending on the clz parameter. PersistenceManagerFactory is passed to Dao, so that, Dao can obtain PersistenceManager from it to interact with the database.

Dao - Dao class, which implements IDao<T>, is a generic Dao. It is generally used select objects like String, Date etc. from the database.

SymbolDao, DataGroupDao and DataDao - even though, generic Dao is capable of handling any object, we use type specific Dao SymbolDao, DataGroupDao and DataDao for Symbol, DataGroup and Data as we do some post processing like filtering.

Let’s go through SymbolDao to understand the reasons for type specific Dao.

in.fins.server.dao.jdo/SymbolDao.java


        @Override
        public T selectById(Class<T> clz, Map<?, ?> parameters)
                        throws PersistenceException {
                PersistenceManager pm = pmf.getPersistenceManager();
                Symbol symbol;
                try {
                        String id = (String) parameters.get("symbolName");
                        Symbol sym = pm.getObjectById(Symbol.class,id);
                        Map<String, String> filterMap =
                            (Map<String, String[]>) parameters.get("filterMap");
                        symbol = DaoHelper.applyFilter(sym, filterMap);
                } catch (PersistenceException e) {
                        log.severe(e.getMessage());
                        throw e;
                } finally {
                        pm.close();
                }
                return (T) symbol;
        }

1

get PersistenceManager (PM) from the factory to interact with database.

2

get symbolName from the parameters.

3

call PersistenceManager.getObjectById(), with parameters Symbol.class and String id, to get a Symbol object whose name is equal to id.
JDO lazily loads persistence objects collection fields. While, it loads simple fields like String, Date etc. immediately, it defers loading of Collection, List, Set etc. until they are accessed. In the returned Symbol object, only name and label are set and List<DataGroup> field is not set. When we start accessing the List, internally JDO fires the SQL to fetch DataGroups and constructs the list.
Further, in JDO persistence object is available only when PM is open. For example, if SymbolService tries to access the Symbol, JDO throws PersistenceException as PM is already closed. Way out of this is to use PersistenceManager.detachCopy() and detachCopyAll() methods. These methods, eagerly load all fields and return a copy of that object which may be used by SymbolService even after PM is closed.
For us, these detach methods are of not much use as they load all the data for all previous days, months and years and for all Facts. It results in Cartesian product of periods and facts which is about 800 facts for a Symbol in sample data. This results into two problems, JDO has to retrieve all the rows from multiple tables which puts unnecessary load on the database and when we send the fully loaded symbol object to the client it consumes quite a bit of network bandwidth. Actually, we require only a small set of about 50 facts for Snapshot page and that too only for latest date in each category. When requesting for data, client had sent a filterMap which contains the required fact keys, and we are going use that to get a slim Symbol object from JDO.

4

get Map<String,String> filterMap from parameters.

5

call DaoHelper.applyFilter() with unfiltered Symbol object and filterMap as parameters. This method access only the required DataGroup and Data, and filters them further using filterMap keys. It creates a new Symbol object, and copies required values from unfiltered Symbol, and returns the new symbol object. In a way, applyFilter() is an optimized equivalent of detachCopy().

6

close PM and return the copy of symbol to caller.

Method applyFilter() accomplishes three things; detaches the object, optimizes the database access and trims down the symbol size. This sort of post processing is not possible with the generic Dao. DataGroupDao and DataDao follow a similar pattern and focus on much smaller set of data.

Detach and RPC

While making copy of persistence objects, we should bear in mind that we are dealing with the enhanced classes where many java types are changed to DataNucleus types. For example, java.util.Date is enhanced to org.datanucleus.sco.simple.Date and in case, we assign the Date from persistence object to the copy, then GWT RPC throws Serialization exception as GWT doesn’t know about DataNucleus date types. Way out is to create a new Date object with copyOfData.setDate(new Date(data.getDate().getTime())) and assign it the copy.

DaoHelper - the last class of in.fins.server.dao.jdo contains post processing static helper methods.

Service

Classes from service layer like SymbolService interacts with DAO layer using abstract DaoFactory and IDao and thus they are totally isolated from underlying ORM layer. Following code snippet from SymbolService shows a typical interaction with DAO.

in.fins.server.dao.jdo/SymbolDao.java

        @Override
        public Symbol getSymbol(String symbolName, Map<String, String[]> filterMap)
                        throws Exception {
                try {
                        initDaoFactory();
                        IDao<Symbol> dao = daoFactory.getDao(Symbol.class);
                        Map<String, Object> parameters =
                                            new HashMap<String, Object>();
                        parameters.put("symbolName", symbolName);
                        parameters.put("filterMap", filterMap);                        return dao.selectById(Symbol.class, parameters);
                } catch (Exception e) {
                        log.warning(e.getMessage());
                        throw e;
                }
        }

1

initialize the DaoFactory which gets the orm name from ServletContext and calls getDaoFactory() method of abstract DaoFactory which returns appropriate DaoFactory based on orm name and returned DaoFactory is assigned to daoFactory field.

2

get instance of Dao and assign it to IDao. DaoFactory return type specific or generic dao based into class parameter.

3

create parameter map and set symbolName and filterMap parameters.

4

call selectById() method IDao to get the result.
 
 
Logging

In server side classes we are using java.util.logging framework as we used the same framework on client side and GAE supports this logging framework.

To output log messages of DataNucleus, Hibernate and MyBatis we find configuring Log4J is much easier than java.util.logging. For that we have to place the log4j.properties in src directory and add log4j jar to war/WEB-INF/lib directory. With all these ORM packages output messages to file dao.log in war/logs dir.

Forward Pointers

Named PMF - refer JDO : Named PersistenceManagerFactory

Detach - refer JDO : Attach/Detach