Home

Awesome

hindex - Secondary Index for HBase

The solution is 100% Java, compatible with Apache HBase 0.94.8, and is open sourced under ASL.

Following capabilities are supported currently.

How it works

HBase Secondary Index is 100% server side implementation with co processors which persists index data in a separate table. Indexing is region wise and custom load balancer co-locates the index table regions with actual table regions.

si1

Server reads the Index specification passed during the table creation and creates the index table. There will be one index table for one user table and all index information for that user table goes into the same index table.

Put Operation

When a row is put into the HBase (user) table, co processors prepare and put the index information in the corresponding index table. Index table rowkey = region startkey + index name + indexed column value + user table rowkey

E.g.:

Table –> tab1 column family –> cf

Index –> idx1, cf1:c1 and idx2, cf1:c2

Index table –> tab1_idx (user table name with suffix “_idx” )

si2

Scan Operation

For a user table scan, co processor creates a scanner on the index table, scans the index data and seeks to exact rows in the user table. These seeks on HFiles are based on rowkey obtained from index data. This will help to skip the blocks where data is not present and sometimes full HFiles may also be skipped.

si5

si4

Usage

Clients need to pass the IndexedHTableDescriptor with the index name and columns while creating the table

IndexedHTableDescriptor htd = new IndexedHTableDescriptor(usertableName);

IndexSpecification iSpec = new IndexSpecification(indexName);

HColumnDescriptor hcd = new HColumnDescriptor(columnFamily);

iSpec.addIndexColumn(hcd, indexColumnQualifier, ValueType.String, 10);

htd.addFamily(hcd);

htd.addIndex(iSpec);

admin.createTable(htd);

No changes required for Puts, Deletes at client side as index operations for the same are internally handled by co-processors

No change in scan code for the client app.

No need to specify the index(s) to be used. Secondary Index implementation finds the best index for Scan by analyzing the filters used for the query.

Source

This repository contains source for Secondary Index support on Apache HBase 0.94.8.

Building from source and testing

Building from source procedure is same as building HBase source hence it requires

Separate test source (secondaryindex\src\test\java\ )is available for running the tests on secondary indexes.

Note

Configure following configurations in hbase-site.xml for using secondary index.

Property

Property

Property

Property

Future Work