Awesome
Store large objects in memcache or others by slicing them.
- uses read_multi for fast access
- returns nil if one slice is missing
- low performance overhead, only uses single read/write if data is below 1MB
Install
gem install large_object_store
Usage
Rails.cache.write("a", "a"*10_000_000) # => false -> oops too large
store = LargeObjectStore.wrap(Rails.cache)
store.write("a", "a"*10_000_000) # => true -> always!
store.read("a").size # => 10_000_000 using multi_get
store.read("b") # => nil
store.fetch("a"){ "something" } # => "something" executes block on miss
store.write("a" * 10_000_000, compress: true) # compress when greater than 16k
store.write("a" * 1000, compress: true, compress_limit: 100) # compress when greater than 100
store.write("a" * 1000, raw: true) # store as string to avoid marshaling overhead
zstd
zstd compression, a modern improvement over the venerable zlib compression algorithm, is supported by passing the zstd
flag when writing items:
store.write("a" * 10_000_000, compress: true, zstd: true)
For backwards compatibility and to enable safe roll-out of the change in working systems, the zstd
flag defaults to false
.
zstd decompression is used when the zstd magic number is detected at the beginning of compressed data, so zstd: true
does not need to be passed when reading/fetching items.
Author
Ana Martinez<br/>
acemacu@gmail.com<br/>
Michael Grosser<br/>
michael@grosser.it<br/>
License: MIT<br/>