Awesome
level-vinyl
leveldb vinyl adapter and blob store. Saves file contents in a content addressable blob store, file metadata in leveldb. Supports globbing, most of the gulp 4.0 options and emits streaming vinyl files.
Jump to: example / usage / vinyl adapter api / levelup api / install / license
why?
Because level-vinyl is a vinyl adapter, you can:
- use 1000+ gulp plugins to transform files
- aggregate files using multiple globs and negation
- do
gulp.src('src/*.png').pipe(db.dest('/assets'))
like a pro:src/1.png
ends up in your database at/assets/1.png
- stream only modified files with
options.since
Because level-vinyl saves metadata (stat, digest and custom properties) to leveldb by a virtual path, you can:
- pipe to and from other databases, local and elsewhere (theoretically).
- index metadata, do map-reduces
- use sublevels to for example, save multiple versions of the same files
- use triggers and hooks to process new files
- or do something else entirely
example
var levelvinyl = require('level-vinyl')
, level = require('level-test')()
, sublevel = require('level-sublevel')
, buffer = require('vinyl-buffer')
, imagemin = require('imagemin-jpegtran')()
, vfs = require('vinyl-fs')
// Create database
var db = level();
var sdb = sublevel(db, { valueEncoding: 'json'});
var vinylDb = levelvinyl(sdb, './example/blobs')
// Create a sublevel for minified images
var min = vinylDb.subvinyl('minified').dest('/')
var main = vinylDb.dest('/')
vfs.src('*.jpg') // file.contents is a buffer
.pipe(main) // save to db
.pipe(imagemin()) // minify
.pipe(min) // save minified to db
Same thing, other way around:
var min = vfs.dest('./example/minified')
var main = vfs.dest('./example')
vinylDb.src('*.jpg') // file.contents is a stream
.pipe(main) // copy to fs
.pipe(buffer()) // imagemin wants buffers
.pipe(imagemin()) // minify
.pipe(min) // copy minified to fs
Note though, it's a levelup database:
vinylDb.get('/example.jpg', function(err, file){
file.contents.pipe(somewhere)
})
vinylDb.get('/example.jpg', { read: false }, function(err, file){
console.log( file.isNull() === true )
})
usage
levelVinyl(db, options || path)
db
a levelup database that has sublevel 6.x.x installedoptions.path || path
where to store the blobs (required)
If you provide an options object, it will be passed to content-addressable-blob-store.
vinyl adapter api
The items below are described in terms of compatibility with vinyl-fs.
db.src(pattern(s)[, options])
Differences
file.contents
is a stream;options.buffer
is not supported. You can use vinyl-buffer to convert streams to buffers (as the above example does).- Only streams regular files (
file.stat.isFile()
is always true).
Features
- aggregate multiple glob patterns, negation
- results are ordered
file.base
is set to the "glob base" oroptions.base
file.isNull()
whenoptions.read == false
- should glob a directory
- return dead stream if globs is empty array
- throw on invalid glob (not a string or array)
- support
options.since
Missing / partial support
- should pass through writes (needs test)
- set glob options (
nobrace
etc).
db.dest([path][, options])
Differences
- The
path
argument is optional and defaults to/
. Files are identified in leveldb by an absolute virtual path, which is constructed from a file'srelative
property, optionally prefixed withpath
. Note thatdest('/docs')
currently does the same asdest('docs')
, because there is no concept of a "current working directory" within the tree. - Saves a small subset of
file.stat
: mtime, ctime, mode and size. Mode is 777 oroptions.mode
; only permission flags are saved.
Features
- updates files after write (
cwd
,base
,path
andstat
) - custom mode with
options.mode
(this is just metadata and has no effect on the blob store) - writes buffers and streams, skips null files
- supports
path
as function (gets a vinyl file, should return a path).
Missing / partial support
- should allow piping multiple dests and should reset streams (needs specific test)
- support
options.cwd
(irrelevant for save, but does setfile.cwd
)
db.watch([pattern(s)][, options][, cb])
Differences
- does not emit a
ready
event andadd()
has no callback argument - the change types
renamed
andadded
are not supported - does not keep the process alive
- both
add()
andremove()
accept glob patterns.
Features
- adds
cb
as change listener options.debounceDelay
: debounce events for the same file, default delay is 500options.maxListeners
is passed to emitter.setMaxListeners- does nothing if
patterns
is empty.
Returns an emitter with these events:
change
with{type, path}
data where type ischanged
ordeleted
.nomatch
, when a changed or deleted file doesn't match the patternsend
when stopped.
And these methods:
add(pattern(s))
: add patterns to be watchedremove(pattern(s))
: exclude files from being matchedend()
: stop watching.
levelup api
db.get(path[, options], cb)
Get a single file. path
will be made absolute and unixified.
db.get('/images/moon.png', cb) // i like this
db.get('images\\moon.png', cb) // but this is fine too
options.read
if false, the blob store isn't accessed andfile.contents
will benull
options.valueEncoding || options.encoding
if set, the retrieved value won't be coerced to Vinyl
db.subvinyl(prefix[, options])
Create a sublevel and install level-vinyl - the sublevel will have its own blob store. Prefix and options are passed to sublevel.
db.put(key, vinyl[, options][, cb])
db.put(vinyl[, options][, cb])
Save a single file.
options.mode
setfile.stat.mode
before saving, default is 777options.base
set the file'sbase
options.valueEncoding || options.encoding
if set, the regularput()
is called.
db.del(key[, cb])
Nothing special, except this again:
db.del('/document') == db.del('\\document')
db.batch(ops[, options], cb)
For put
operations:
options.mode
setfile.stat.mode
on all filesop.mode
same, but per operation (overridesoptions.mode
):
db.batch([{
type: 'put',
key: '/project/readme',
value: readmeFile,
mode: 0755
}])
db.createReadStream([options])
This is a plain stream, i.e. it does not decode values to Vinyl files.
options.since
stream only files modified since a date or timestamp. Ignored for key streams (whenoptions.values === false
).
db.getBlobStore()
Get the blob store instance.
install
With npm do:
npm install level-vinyl