NativeHandler Class Reference

Collaboration diagram for NativeHandler:

Collaboration graph
[legend]

List of all members.

Static Public Member Functions

static Object decode (byte[] b, int flag) throws Exception
static byte[] encode (Object value) throws Exception
static int getMarkerFlag (Object value)
static boolean isHandled (Object value)

Static Protected Member Functions

static Boolean decodeBoolean (byte[] b)
static Byte decodeByte (byte[] b)
static byte[] decodeByteArr (byte[] b)
static Character decodeCharacter (byte[] b)
static Date decodeDate (byte[] b)
static Double decodeDouble (byte[] b) throws Exception
static Float decodeFloat (byte[] b) throws Exception
static Integer decodeInteger (byte[] b)
static Long decodeLong (byte[] b) throws Exception
static Short decodeShort (byte[] b) throws Exception
static String decodeString (byte[] b) throws Exception
static StringBuffer decodeStringBuffer (byte[] b) throws Exception
static StringBuilder decodeStringBuilder (byte[] b) throws Exception
static byte[] encode (byte[] value)
static byte[] encode (StringBuilder value) throws Exception
static byte[] encode (double value) throws Exception
static byte[] encode (Short value) throws Exception
static byte[] encode (float value) throws Exception
static byte[] encode (StringBuffer value) throws Exception
static byte[] encode (String value) throws Exception
static byte[] encode (Character value)
static byte[] encode (Date value)
static byte[] encode (long value) throws Exception
static byte[] encode (int value)
static byte[] encode (Boolean value)
static byte[] encode (Byte value)
static byte[] getBytes (int value)
static byte[] getBytes (long value)
static int toInt (byte[] b)
static long toLong (byte[] b)

Static Private Attributes

static Logger log

Detailed Description

Handle encoding standard Java types directly which can result in significant memory savings:

Currently the Memcached driver for Java supports the setSerialize() option. This can increase performance in some situations but has a few issues:

Code that performs class casting will throw ClassCastExceptions when setSerialize is enabled. For example:

mc.set( "foo", new Integer( 1 ) ); Integer output = (Integer)mc.get("foo");

Will work just file when setSerialize is true but when its false will just throw a ClassCastException.

Also internally it doesn't support Boolean and since toString is called wastes a lot of memory and causes additional performance issue. For example an Integer can take anywhere from 1 byte to 10 bytes.

Due to the way the memcached slab allocator works it seems like a LOT of wasted memory to store primitive types as serialized objects (from a performance and memory perspective). In our applications we have millions of small objects and wasted memory would become a big problem.

For example a Serialized Boolean takes 47 bytes which means it will fit into the 64byte LRU. Using 1 byte means it will fit into the 8 byte LRU thus saving 8x the memory. This also saves the CPU performance since we don't have to serialize bytes back and forth and we can compute the byte[] value directly.

One problem would be when the user calls get() because doing so would require the app to know the type of the object stored as a bytearray inside memcached (since the user will probably cast).

If we assume the basic types are interned we could use the first byte as the type with the remaining bytes as the value. Then on get() we could read the first byte to determine the type and then construct the correct object for it. This would prevent the ClassCastException I talked about above.

We could remove the setSerialize() option and just assume that standard VM types are always internd in this manner.

mc.set( "foo", new Boolean.TRUE ); Boolean b = (Boolean)mc.get( "foo" );

And the type casts would work because internally we would create a new Boolean to return back to the client.

This would reduce memory footprint and allow for a virtual implementation of the Externalizable interface which is much faster than Serialzation.

Currently the memory improvements would be:

java.lang.Boolean - 8x performance improvement (now just two bytes) java.lang.Integer - 16x performance improvement (now just 5 bytes)

Most of the other primitive types would benefit from this optimization. java.lang.Character being another obvious example.

I know it seems like I'm being really picky here but for our application I'd save 1G of memory right off the bat. We'd go down from 1.152G of memory used down to 144M of memory used which is much better IMO.

http://java.sun.com/docs/books/tutorial/native1.1/integrating/types.html

Author:
Kevin A. Burton

Greg Whalin <greg@meetup.com>

Member Function Documentation

static Object decode ( byte[]  b,
int  flag 
) throws Exception [static]

Decodes byte array using memcache flag to determine type.

Parameters:
b 
marker 
Returns:
Exceptions:
Exception 

static Boolean decodeBoolean ( byte[]  b  )  [static, protected]

static Byte decodeByte ( byte[]  b  )  [static, protected]

static byte [] decodeByteArr ( byte[]  b  )  [static, protected]

static Character decodeCharacter ( byte[]  b  )  [static, protected]

static Date decodeDate ( byte[]  b  )  [static, protected]

static Double decodeDouble ( byte[]  b  )  throws Exception [static, protected]

static Float decodeFloat ( byte[]  b  )  throws Exception [static, protected]

static Integer decodeInteger ( byte[]  b  )  [static, protected]

static Long decodeLong ( byte[]  b  )  throws Exception [static, protected]

static Short decodeShort ( byte[]  b  )  throws Exception [static, protected]

static String decodeString ( byte[]  b  )  throws Exception [static, protected]

static StringBuffer decodeStringBuffer ( byte[]  b  )  throws Exception [static, protected]

static StringBuilder decodeStringBuilder ( byte[]  b  )  throws Exception [static, protected]

static byte [] encode ( byte[]  value  )  [static, protected]

static byte [] encode ( StringBuilder  value  )  throws Exception [static, protected]

static byte [] encode ( double  value  )  throws Exception [static, protected]

static byte [] encode ( Short  value  )  throws Exception [static, protected]

static byte [] encode ( float  value  )  throws Exception [static, protected]

static byte [] encode ( StringBuffer  value  )  throws Exception [static, protected]

static byte [] encode ( String  value  )  throws Exception [static, protected]

static byte [] encode ( Character  value  )  [static, protected]

static byte [] encode ( Date  value  )  [static, protected]

static byte [] encode ( long  value  )  throws Exception [static, protected]

static byte [] encode ( int  value  )  [static, protected]

static byte [] encode ( Boolean  value  )  [static, protected]

static byte [] encode ( Byte  value  )  [static, protected]

static byte [] encode ( Object  value  )  throws Exception [static]

Encodes supported types

Parameters:
value Object to encode.
Returns:
byte array
Exceptions:
Exception If fail to encode.

static byte [] getBytes ( int  value  )  [static, protected]

static byte [] getBytes ( long  value  )  [static, protected]

static int getMarkerFlag ( Object  value  )  [static]

Returns the flag for marking the type of the byte array.

Parameters:
value Object we are storing.
Returns:
int marker

static boolean isHandled ( Object  value  )  [static]

Detemine of object can be natively serialized by this class.

Parameters:
value Object to test.
Returns:
true/false

static int toInt ( byte[]  b  )  [static, protected]

This works by taking each of the bit patterns and converting them to ints taking into account 2s complement and then adding them..

Parameters:
b 
Returns:

static long toLong ( byte[]  b  )  [static, protected]

Member Data Documentation

Logger log [static, private]

Initial value:

                Logger.getLogger( NativeHandler.class.getName() )

The documentation for this class was generated from the following file:
Generated on Sat May 26 06:42:31 2012 for Project Wonder by  doxygen 1.5.8