Home

Awesome

NP Dev Collections

Foundational code for NP Dev Collections was generously donated by PixStori, an aural history social media tool built on Meteor. Tell your stori.

NPDev Collections combines a mix of technologies to facilitate the creation of offline first collections, data over methods with pagination, and support for SSR out of the box.

Install with:

$ meteor add npdev:collections

Check out my starter for a complete example site using NP Dev Collections and other server-render tools.

Why

Basically, I like Mongo and MiniMongo for it's low bar for entry, and it's simple iteration. It's great for starting new projects and prototyping early and rapidly. I wanted to create something that added a few of the modern touches, like offline-first data, and server side rendering, while retaining that wonderful simplicity as much as possible.

What

There are a lot moving parts involved getting all the parts working together. Here are some of the hurdles:

That's a lot of stuff to keep track of, and it's kind of a pain to do it manually. NPDev;Collections keep track of all that for you, and provides the tools to easily set it all up.

How

Quick Start!

Install with:

meteor add npdev:collections

We also need react and react-dom npm packages of course. These are not defined in this package, so that the version can be kept up to date in the main project. This package requires a version of react which contains support for hooks - 16.8+.

The first thing we need are our collections. They can be created using the simple createCollection method. All this is does is create a Meteor Collection on the server, or a Ground:DB server on the client, and connect the schema using aldeed:collection2's attachSchema (on the server). It's necessary to use this method to create your collections to register them with NPDev:Collections. I'll probably add some facilities to allow more granular control over these in the future. Of course, these collections should all be included in the server bundle.

import { createCollection } from 'meteor/npdev:collections'
import { CommentSchema } from './CommentSchema'

const Comments = createCollection('comments', CommentSchema)

export default Comments

Basic use requires the creation of custom hooks for each query you want to set up. The createConnector utility function accept a set of properties - name, collection, an isomorphic validation method, and an isomorphic query generator. This API is inspired by, and builds on the API of mdg:validated-method. Here is an example from PixStori:

import { createListHood } from 'meteor/npdev:collections'

// getPublicQuery builds a query which selects the appropriate public documents
const getPublicQuery = () => ({
  public: true
})

export const useComments = createConnector({
  name: 'tiles',
  collection: Comments,
  // This runs on client and server, and in both methods and SSR contexts.
  validate () {},
  // So does this! Be careful with security.
  query () {
    return getPublicQuery()
  }
})

export const useGroupComments = createConnector({
  name: 'groupComments',
  collection: Comments,
  validate ({ groupId }) {
    // Here we could use SimpleSchema and/or throw a validated-error, etc.
    // See the ValidateMethod documentation for more.
    check(groupId, String)
  },
  query: ({ groupId }) => ({
    $and: [
      { groupId },
      getPublicQuery()
    ]
  })
})

NOTE: These hooks must be included in the server build, not just in the react tree, but somewhere statically, so they can set up the necessary methods. They don't need to be included statically in the client bundle, which allows for code splitting using the dynamic-import package.

Using this, along with createConnector, it sets up everything we need on the server, and on the client to do offline-first, data-over-methods, with pagination, and SSR, with data hydration, etc. (along with using a set of providers in SSR and hydration code). Super spiffy! In use, it looks like this:

// Here we use the group comments.
const GroupFeedPage = ({ limit, offset, order, orderBy, groupId }) => {
  const { groupComments, groupCommentsAreLoading } = useGroupComments({ groupId, limit, offset, order, orderBy })
  return <FeedPage tiles={groupComments} isLoading={groupCommentsAreLoading} />
}

That's already a pretty easy way to grab data! But we also want to have SSR with data hydration.

import React from 'react'
import { StaticRouter } from 'react-router'
import { renderToString } from 'react-dom/server'
import { onPageLoad } from 'meteor/server-render'
import App from '/imports/App'
import { DataCaptureProvider } from 'meteor/npdev:collections'

onPageLoad(sink => {
  const context = {}

  // use the DataCaptureProvider with a scoped dataHandle
  const dataHandle = {}
  const app = <DataCaptureProvider handle={dataHandle}>
    <StaticRouter location={sink.request.url} context={context}>
      <App />
    </StaticRouter>
  </DataCaptureProvider>

  // render the app to html
  const content = renderToString(app)

  // render out the html
  sink.renderIntoElementById('root', content)

  // render out the captured data
  sink.appendToBody(dataHandle.toScriptTag())
})

Behind the scenes this provider is watching for all the queries that happen during rendering of the current route, and captures that data in a property on dataHandle. Then the toScriptTag method is used to render out a <script> tag which contains an EJSON encoded copy of all that data. This will be hydrated on the client side, like so:

import { onPageLoad } from 'meteor/server-render'

onPageLoad(sink => {
  import React from 'react'
  import { hydrate } from 'react-dom'
  import { BrowserRouter } from 'react-router-dom'
  import { DataHydrationProvider, hydrateData } from 'meteor/npdev:collections'
  import App from '/imports/App'

  // Load the data into offline storage
  hydrateData()

  // The isHydrating flag basically tells the hooks not to bother loading data
  // over methods, since we just hydrated all the data above.
  const hydrationHandle = { isHydrating: true }

  const app = <DataHydrationProvider handle={hydrationHandle}>
    <BrowserRouter>
      <App />
    </BrowserRouter>
  </DataHydrationProvider>

  // hydrate the app using React.hydrate (<div id="root"></div> defined using
  // static-html package in /client/main.html)
  hydrate(app, document.getElementById('root'), () => {
    // set the isHydrating flag to false, so that subsequent renders will know
    // to fetch data.
    hydrationHandle.isHydrating = false
  })
})

TODOS:

You can find some of the SSR stuff here (without the data part for now), in my starter repo.