Android: Using DatabaseUtils.InsertHelper for faster insertions into SQLite database

AndroidOS includes the DatabaseUtils.InsertHelper class for speeding up insertions into an SQLite database. However, very little documentation or examples seem to be available to show how to use this class. I hope this post will help to make InsertHelper a little less mysterious than it apparently has been.

It’s often the case that bulk insertions are performed in the onCreate method of an SQLiteOpenHelper, so I’ll use that context for this example. (This has implications on transaction management, which in turn affects performance, as described below.)

Suppose the onCreate method currently looks something like this:

private class DatabaseHelper extends SQLiteOpenHelper {
    @Override
    public void onCreate(SQLiteDatabase db) {
        ContentValues values = new ContentValues();
        while (moreRowsToInsert) {
            // ... create the data for this row (not shown) ...

            // Add the data for each column
            values.put("Greek", greekData);
            values.put("Ionic", ionicData);
            // ...
            values.put("Roman", romanData);

            // Insert the row into the database.
            db.insert("columnTable", null, values);
        }
    }
    //...
}

Using DatabaseUtils.InsertHelper, this would be re-written as:

import android.database.DatabaseUtils.InsertHelper;
//...
private class DatabaseHelper extends SQLiteOpenHelper {
    @Override
    public void onCreate(SQLiteDatabase db) {
        // Create a single InsertHelper to handle this set of insertions.
        InsertHelper ih = new InsertHelper(db, "columnTable");

        // Get the numeric indexes for each of the columns that we're updating
        final int greekColumn = ih.getColumnIndex("Greek");
        final int ionicColumn = ih.getColumnIndex("Ionic");
        //...
        final int romanColumn = ih.getColumnIndex("Roman");

        try {
            while (moreRowsToInsert) {
                // ... Create the data for this row (not shown) ...

                // Get the InsertHelper ready to insert a single row
                ih.prepareForInsert();

                // Add the data for each column
                ih.bind(greekColumn, greekData);
                ih.bind(ionicColumn, ionicData);
                //...
                ih.bind(romanColumn, romanData);

                // Insert the row into the database.
                ih.execute();
            }
        }
        finally {
            ih.close();  // See comment below from Stefan Anca
        }
    }
    //...
}

As this shows, using InsertHelper is barely more complicated than using SQLiteDatabase.insert. The major differences are that you need to call ih.prepareForInsert() before adding (“binding”) the column data; and you need to obtain each column’s numeric index, which we get by calling ih.getColumnIndex() prior to the loop.

After replacing SQLiteDatabase.insert with DatabaseUtils.InsertHelper, the database insertion speed went from the equivalent of about 95 rows per second to about 525 rows per second. (“Equivalent of”, because the app also spends cycles creating the data to insert. Here, the performance timings are measured using constant data, eliminating that overhead.)

Other ways to speed up insertions

In addition to that gain, two more changes then brought the insertion speed to well over 900 rows per second. Whether these tricks techniques work for you will depend on your application.

Don’t bind empty columns

In my app, the data for at least 50% of the columns is empty. By skipping the call to ih.bind() when the column data is a null or empty string, I saw a roughly 30% performance boost.

Temporarily disable database thread locking

I’m loading the database during the onCreate method of my app’s SQLiteOpenHelper. During this time, it seems safe to assume that only one thread is accessing the database, so I use SQLiteDatabase.setLockingEnabled() to temporarily (let me emphasize that: temporarily) disable thread locks within the database API. This yielded about a 35% performance gain:

    public void onCreate(SQLiteDatabase db) {
    //...
    try {
        // *Temporarily* (have I emphasized that enough?) disable
        // thread locking in the database. Be sure to re-enable locking 
        // within a finally block.
        db.setLockingEnabled(false);
        // ... load the database ...
    }
    finally {
        db.setLockingEnabled(true);
    }

Transactions and performance

A number of people have cited performance gains through use of explicit transactions in SQLite. However, SQLiteOpenHelper creates a transaction before invoking its callback methods (onCreate, onUpgrade, and onOpen), so explicit transaction control is unnecessary within those methods. (SQLiteOpenHelper will assume that the transaction was successful unless your method throws an exception.)

You would need to manage your own transactions if your insertion code is running outside of one of SQLiteOpenHelper‘s callback methods. The main APIs for this are SQLiteDatabase.beginTransaction, SQLiteDatabase.setTransactionSuccessful, and SQLiteDatabase.endTransaction.

It is possible to nest transactions, but, not surprisingly, this doesn’t seem to help performance. In fact, I saw a very slight performance degradation when using nested transactions (approximately 1%, probably below the accuracy of the measurements.) I also tried periodically closing the current transaction — the first of these being the transaction that was opened by SQLiteOpenHelper — then opening a new one. This didn’t yield much improvement, if any.