Using ODB on Mobile and Embedded Systems

• 1 Building SQLite and ODB Runtime Libraries
• 2 Building the Example Using Static Libraries
• 3 Building the Example Using Shared Libraries
• 4 Minimizing Runtime and Generated Code Size

This guide discusses a number of topics relevant to using ODB on mobile
and embedded systems. Specifically, it shows how to cross-compile the
ODB runtime libraries in both static and shared variants. It also
provides an indication of footprint as well as ways to minimize the
generated code size.

While this guide uses the Raspberry Pi
ARM GNU/Linux computer as a sample target, other mobile/embedded
platforms will normally require very similar steps. We will also
explicitly mention a few areas that may need platform-specific changes.
If you run into any problems while trying to apply the below steps for
your platform, feel free to ask for help on the odb-users mailing list. For an Android target, refer to Using ODB on Android.

Some mobile and embedded targets support compilation on the
target itself (the so-called self-hosted development). However, due to
various constraints (RAM, CPU, connectivity, etc.), it is rarely
practical to develop any real-world application on the target directly.
As a result, a cross-compilation environment is normally used instead
and in this guide we will only consider this approach.

Note also that this guide is not an introduction to developing
C++ applications for mobile/embedded systems. In particular, it assumes
that you are familiar with the concept of cross-compilation and how it
works for your target. Before proceeding further with this guide, it is
recommended that you install the cross-compiler for your target on your
development machine, build a simple "Hello, World" C++ application, copy
it on to your target, and make sure it runs correctly, that is, there
are no missing libraries, crashes, etc.

In this guide we will use the arm-bcm2708hardfp-linux-gnueabi cross-compiler toolchain from the official Raspberry Pi tools repository. However, other toolchains can be used as well, such as the Debian/Ubuntu ARM cross-compiler package.

The most popular choice of a database for mobile/embedded systems
is SQLite. So in this guide we will build the "Hello, World" example
from the odb-examples package for the Raspberry Pi target
using SQLite as the database. This, however, is not to say that other
databases (or even multiple database at once) cannot be used on
mobile/embedded systems. All we will need to do is cross-compile or get
an already pre-built client library for the database(s) we want to use.

The following list provides a high-level overview of the steps we
will need to perform in order to build an ODB-based application for our
mobile/embedded target. The comments in brackets indicate whether this
step is performed on the development machine or on the target.

1. Install the ODB compiler [development machine]
2. Cross-compile the database (client) library and ODB runtimes [development machine]
3. Compile headers with the ODB compiler [development machine]
4. Cross-compile the generated code and application code [development machine]
5. Copy the application onto the target and run [target]

The easiest way to install the ODB compiler is to use the
pre-compiled binary package for your development machine. For example,
if your development machine is x86-64 GNU/Linux, then you will use the odb-X.Y.Z-x86_64-linux-gnu
package. You can also build the ODB compiler from source code if you
wish (note that in this case you will need to build it for your
development machine, not for your mobile/embedded target).

The next step is to cross-compile the SQLite database (or another
database client library) and the ODB runtimes libraries. Here we have
two options: we can build them as static or as shared libraries. When it
comes to mobile/embedded systems, static libraries have a number of
advantages. Firstly, when using static libraries, the resulting
executable will only contain ODB runtime code that is used by the
application. This minimizes the executable size (both on disk and in
RAM). Static libraries are also easier to work with; they are easier to
build, to link to, and they don't need to be deployed to the target.
Shared libraries in the mobile/embedded context may have an advantage if
more than one application is using them. As a result, we recommend
that you use static libraries unless you have multiple applications that
use ODB and even in this case it makes sense to actually check if there
are any footprint savings. If you decide to build and use shared
libraries, we strongly recommend that you first try to build static
variants to make sure that everything works in that simpler case.

Note also that it is possible to use a shared library for the
database and static libraries for the ODB runtimes. This would make
sense, for example, if SQLite came pre-installed (as a shared library)
on your target because it is used by other applications.

As the first step, choose a working directory where we will build everything. Let's also create the install
sub-directory in this working directory; this is the place where we
will install headers, libraries, etc., that we are going to build (note
that it's a bad idea to install them into, say, /usr/local because these libraries will be built for our target, not for the development machine).

Let's also assume that our cross-compiler toolchain and the ODB
compiler are also in this working directory. That is, we have the arm-bcm2708hardfp-linux-gnueabi sub-directory (cross-compiler for Raspberry Pi) and the odb-X.Y.Z-x86_64-linux-gnu or similar sub-directory (ODB compiler). Let's also rename odb-X.Y.Z-x86_64-linux-gnu to just odb
for easier referencing. Note that the cross-compiler and the ODB
compiler don't really have to be in our working directory. If you have
them installed somewhere else, simply adjust the paths in the
instructions below.

Let's also add the cross-compiler to the PATH environment variable. That is, in a terminal where you will perform the compilation, do:

   export PATH=`pwd`/arm-bcm2708hardfp-linux-gnueabi/bin:$PATH
   arm-bcm2708hardfp-linux-gnueabi-g++ --version

The second command verifies that the C++ cross-compiler can now be executed directly by printing its version.

Another step that we need to perform before we can start building is to verify that the cross-compiler toolchain doesn't have broken .la files, specifically libstdc++.la and, if you are using a pre-built SQLite or another database (client) library, the .la file for that library. The .la files often end up broken because of the different directories where the toolchain was initially installed (by whomever built it) and where you unpacked it. The .la files will almost always end up broken if you installed your toolchain by simply unpacking a .tar.gz archive. On the other hand, if you built the toolchain yourself (and didn't move the installation directory) or if you installed it from a package (e.g., from Debian/Ubuntu) then the .la files are most likely intact and you can skip this step.

To verify that the .la files are correct, search for libstdc++.la in the toolchain directory. If none is found, then you don't need to do anything. If one is found, open it in a text editor and search for the libdir variable (usually right at the bottom). The value of this variable should be the path to the directory where the libstdc++.la file resides. If it is not valid, correct it and save the file. If you are using a pre-built SQLite or another database (client) library, repeat these steps for its .la file.

The next section discusses building the database and the ODB runtimes as either static or shared libraries. The two sections after that show how to build and run the hello example using static or shared libraries, respectively. The final section in this guide provides an indication of the resulting binary sizes as well as covers various ways to minimize the generated code size.

Building SQLite and ODB Runtime Libraries

Unless you already have SQLite built for your target, let's first do that. Download the sqlite-autoconf-XYZ.tar.gz archive and unpack it into the working directory:

   tar xfz sqlite-autoconf-XYZ.tar.gz
   cd sqlite-autoconf-XYZ

Next we need to configure SQLite for our target. Here you may need to add additional C compiler flags that are specific to your target. For instance, if you are using a generic ARM toolchain (such as from from Debian/Ubuntu) instead of the Raspberry Pi-specific one, then you may need to specify the CPU version, etc., explicitly by adding the -march=armv6 -mfpu=vfp -mfloat-abi=hard options to the CFLAGS (and later CXXFLAGS) variable. You may also want to adjust the optimization level. Note also that the value of the --host option must match the cross-compiler tool prefix (that is arm-bcm2708hardfp-linux-gnueabi in arm-bcm2708hardfp-linux-gnueabi-g++) exactly. The basic configuration for a static SQLite library looks like this:

   ./configure CFLAGS="-Os -DSQLITE_ENABLE_UNLOCK_NOTIFY=1" --disable-shared --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

And for a shared SQLite library — like this:

   ./configure CFLAGS="-Os -DSQLITE_ENABLE_UNLOCK_NOTIFY=1" --disable-static --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

If here or below a configure script terminates with an error, then you can find more detailed information about the cause of the error in the config.log file created by this script.

If, however, there are no errors during the configuration, then the next step is to build and install SQLite:

   make
   make install

After this command you should have the SQLite headers and static/shared library in the install sub-directory of the working directory. Here and below you can also use the install-strip target to strip the installed libraries.

Once SQLite is ready, we can move on to building the ODB runtimes. The minimum that you will need is libodb and libodb-sqlite. If you also want to use one of the profile libraries, then you can build it in the same way.

To build libodb, unpack its source code into the working directory:

   tar xfz libodb-X.Y.Z.tar.gz
   cd libodb-X.Y.Z

The configuration and building steps are similar to SQLite. Again, don't forget to add any extra compiler options to CXXFLAGS if your target requires them. Static library:

   ./configure CXXFLAGS="-Os" --disable-shared --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Shared library:

   ./configure CXXFLAGS="-Os" --disable-static --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Once configuration is done:

   make
   make install

To build libodb-sqlite, unpack its source code into the working directory:

   tar xfz libodb-sqlite-X.Y.Z.tar.gz
   cd libodb-sqlite-X.Y.Z

Then configure and build. Static library:

   ./configure CXXFLAGS="-Os" CPPFLAGS="-I`pwd`/../install/include" LDFLAGS="-L`pwd`/../install/lib" --disable-shared --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Shared library:

   ./configure CXXFLAGS="-Os" CPPFLAGS="-I`pwd`/../install/include" LDFLAGS="-L`pwd`/../install/lib" --disable-static --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Once configuration is done:

   make
   make install

Building the Example Using Static Libraries

Once the static libraries for SQLite and the ODB runtimes are ready, we can build the hello example. First, unpack the odb-examples package:

   tar xfz odb-examples-X.Y.Z.tar.gz
   cd odb-examples-X.Y.Z/hello

While it is possible to use autotools-based build system to cross-compile the examples, in this guide we will use modified manual build instructions from the accompanying README file. First, we compile the person.hxx header with the ODB compiler:

    ../../odb/bin/odb -d sqlite --generate-query --generate-schema person.hxx

Next we build everything with the cross-compiler:

   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c person-odb.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c -DDATABASE_SQLITE driver.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -L../../install/lib -o driver driver.o person-odb.o -lodb-sqlite -lodb -lsqlite3 -lpthread -ldl

The result of the last command is the driver executable which we can copy over on to the target and run:

   raspberrypi $ uname -a
   Linux raspberrypi 3.2.27+ #3 PREEMPT Sat Dec 15 18:52:34 SAST 2012 armv6l GNU/Linux
   
   raspberrypi $ ./driver --database /tmp/test.db
   Hello, John Doe!
   Hello, Jane Doe!
   
   count  : 3
   min age: 31
   max age: 33

Building the Example Using Shared Libraries

Once the shared libraries for SQLite and the ODB runtimes are ready, we can build the hello example. First, unpack the odb-examples package:

   tar xfz odb-examples-X.Y.Z.tar.gz
   cd odb-examples-X.Y.Z/hello

While it is possible to use autotools-based build system to cross-compile the examples, in this guide we will use modified manual build instructions from the accompanying README file. First, we compile the person.hxx header with the ODB compiler:

    ../../odb/bin/odb -d sqlite --generate-query --generate-schema person.hxx

Next we build everything with the cross-compiler:

   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c person-odb.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c -DDATABASE_SQLITE driver.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -L../../install/lib -o driver driver.o person-odb.o -lodb-sqlite -lodb -lsqlite3

The result of the last command is the driver executable. To run it on the target, besides the executable itself, we will also need to copy over the shared libraries from the install/lib sub-directory. The libraries that we will need are: libsqlite3.so.*, libodb-X.Y.so, and libodb-sqlite-X.Y.so. On the target we will also need to make sure that the dynamic linker can find them. While how exactly to achieve this depends on the target, installing them into /usr/local/lib or adding their directory to the LD_LIBRARY_PATH environment variable will work for Raspberry Pi or any other GNU/Linux derivative. For example, assuming that our executable and all the libraries are in the current directory (on the target), we can run the example like this:

   raspberrypi $ uname -a
   Linux raspberrypi 3.2.27+ #3 PREEMPT Sat Dec 15 18:52:34 SAST 2012 armv6l GNU/Linux
   
   raspberrypi $ export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH
   raspberrypi $ ./driver --database /tmp/test.db
   Hello, John Doe!
   Hello, Jane Doe!
   
   count  : 3
   min age: 31
   max age: 33

Minimizing Runtime and Generated Code Size

To give an indication of the footprint one can expect when using ODB on mobile/embedded systems, the stripped hello example executable when built with static SQLite and ODB runtime libraries is 533Kb. While this may seem like a lot for such a simple application, keep in mind that it includes the whole SQLite database as well as the libodb and libodb-sqlite runtimes, all of which are "once-off costs", that is, they don't change with the number of persistent classes used by the application.

A more useful size breakdown can be obtained from the build that uses shared libraries (again, all stripped):

   35348  driver
   38244  libodb-2.1.so
   107532 libodb-sqlite-2.1.so
   504232 libsqlite3.so.0.8.6

As you can see, the driver itself, which contains the generated code for one persistent class and one view, is only 34Kb. Note also that the combined size of all the libraries and the executable (669Kb) is about 25% greater than that of the static executable.

The ODB compiler implements fine-grained control over various features provided by the generated code. By not enabling functionality that is not needed by your application you can greatly reduce the generated code size and the resulting application footprint. In particular, the following ODB command line flags control optional functionality provided by the generated code:

   --generate-query
   --generate-prepared
   --omit-unprepared
   --generate-session
   --generate-schema

Also, when generating database schema embedded into the C++ code, it can be wasteful to keep that code in the main executable (and thus in the device's memory) all the time. For example, if you need to create the database schema only once when the device is first turned on, then it may make sense to factor the schema creation function into a separate executable. This can be achieved with the --schema-format separate option which instructs the ODB compiler to generate the schema creation C++ code into a separate source file.

If your application is single-threaded, then it is also possible to slightly reduce the ODB runtime sizes by disabling multi-threading support. To achieve this, pass the --disable-threads option to the configure commands.

To reduce runtime memory usage and dynamic memory allocations in persistent classes, consider using char[N] arrays (or C++11 std::array<char,N>) instead of std::string or similar for representing string and binary data. For example:

   #pragma db object
   class person
   {
     ...
     
     char first[32];
     char last[32];
     
     #pragma db type("BLOB")
     char public_key[1024];
   };