Home

Awesome

Firestorm

icon

Makeshift Cloud Firestore C# API that works on Unity via REST API, by UnityWebRequest that can ensure cross-platform support. Only basic functions implemented. "Makeshift" means I wrote everything very hurriedly and the performance is really bad. I am ready to deprecate all of this when the real thing came out.

Status

This is far from identical with the real C# API

For usage of the real thing please see : https://jskeet.github.io/google-cloud-dotnet/docs/Google.Cloud.Firestore.Data/datamodel.html and you know how much you have to migrate after that thing came out.

Why Cloud Firestore

Approach

Requires

I put the requirement as an "assembly override" in the asmdef explicitly. It requires 3 dll total :

For the test assembly, it requires Cloud Function also. I opt to hide Admin SDK on the server and let the cloud function handle test resets.

LitJSON is baked in the package. It is literally "little" that I could embed it in with my modifications. (60KB)

Receiving data with LitJSON

After you got the document snapshot, snapshot.ConvertTo<T> will change JSON into your C# data container. How that works is according to LitJSON and may not be the same as the real upcoming SDK. Things to watch out :

Why not Unity's JsonUtility

It sucks! The JSON from Firestore has polymorphic union fields (see example) and it is impossible to work with without good iteration method on the JSON. LitJSON and Json.NET could iterate through json (with JsonData and JObject respectively) and also build a hand-made json from scratch.

Why not Newtonsoft Json.NET

It might be top-quality fast and reliable thanks to millions of users, but at its core it uses DynamicMethod. It does not work on platform like Android. If you use it, at the end you might encounter :

  at Newtonsoft.Json.Utilities.DynamicReflectionDelegateFactory.CreateDynamicMethod (System.String name, System.Type returnType, System.Type[] parameterTypes, System.Type owner) [0x00000] in /_/Src/Newtonsoft.Json/Utilities/DynamicReflectionDelegateFactory.cs:45 

  at Newtonsoft.Json.Utilities.DynamicReflectionDelegateFactory.CreateDefaultConstructor[T] (System.Type type) [0x00000] in /_/Src/Newtonsoft.Json/Utilities/DynamicReflectionDelegateFactory.cs:244

And if you look at the source you can see a lot of DynamicMethod usage. From what I see it is trying to synthesize constructor to even a concrete type that I have everything defined beforehand. So looks like no escape.

Limitations

I made this just enough to adopt Firestore as soon as possible. Features are at bare minimum. Code is a mess and also performance is really BAD. (Sent JSON are even indented just so that debugging would be easy..)

How to use

Please look in the test assembly folder for some general ideas, I don't have time to write a guide yet.. but it always begin with something like Firestorm.Collection("c1").Document("d1").Collection("c1-1").Document("d2")._____. (Use FirebaseAuth.DefaultInstance to sign in first! It works on the CurrentUser.)

When migrating to the real thing later, Firestorm would become FirestoreDatabase instance you get from somewhere. Everything else should be roughly the same. (?)

How to run tests/to make sure it works

You will want to be able to pass all tests as database is a sensitive thing and could wreck your game if not careful. (Or if I made mistake somewhere)

The test will run against your real Firebase account and cost real money as it writes and cleans up the Firestore on every test (but probably not much). There are things that is required to setup beforehand.


import * as functions from 'firebase-functions'
import * as admin from 'firebase-admin'
import { HttpsError } from 'firebase-functions/lib/providers/https'
admin.initializeApp()

function testSecretCheck(testSecret: string) {
    if (testSecret !== "notasecret") {
        throw new HttpsError("internal", `Your test secret ${testSecret} is incorrect.`);
    }
}

/**
 * Used for unit testing. Delete and recreate user on every test.
 * @param recreateUser This is true on [TearDown] in C# so that it just delete the user and not create back. After a test there should not be any test user left.
 */
async function ensureFreshUser(email: string, password: string, recreateUser: boolean) {
    try {
        const superUser: admin.auth.UserRecord = await admin.auth().getUserByEmail(email)
        //If the user exist delete him.
        await admin.auth().deleteUser(superUser.uid)
    }
    catch (e) {
        if (e.code !== "auth/user-not-found") {
            throw e
        }
        //Does not exist, it is fine.
    }
    if (recreateUser === false) {
        await admin.auth().createUser({ email: email, password: password })
    }
}

export const firestormTestCleanUp = functions.https.onCall(async (data, context) => {

    const testCollectionName: string = "firestorm-test-collection"

    const testDataName1: string = "firestorm-test-data-1"
    const testDataName2: string = "firestorm-test-data-2"
    const testDataName3: string = "firestorm-test-data-3"

    const testSubCollectionName: string = "firestorm-test-sub-collection"
    const testDataName21: string = "firestorm-test-data-21"
    const testDataName22: string = "firestorm-test-data-22"

    try {
        testSecretCheck(data.testSecret)
        await Promise.all([
            ensureFreshUser(data.superUserId, data.superUserPassword, data.recreateUser),
            //No need to demolish everything, the test uses just these 5 documents.
            admin.firestore().collection(testCollectionName).doc(testDataName1).delete(),
            admin.firestore().collection(testCollectionName).doc(testDataName2).delete(),
            admin.firestore().collection(testCollectionName).doc(testDataName3).delete(),
            admin.firestore().collection(testCollectionName).doc(testDataName2).collection(testSubCollectionName).doc(testDataName21).delete(),
            admin.firestore().collection(testCollectionName).doc(testDataName2).collection(testSubCollectionName).doc(testDataName22).delete(),
        ])
    } catch (error) {
        throw new HttpsError("internal", `${error.code} -> ${error}`)
    }
});

Notice testSecretCheck method, you can change the password to match what's in your FirestormConfig. Every time you run each test this cloud function will run 2 times at set up and at tear down. (Costing you small amount of money)

Play Mode test

Change the asmdef from Editor only to all platforms. Try and see if all the test can run successfully in play mode. It is a bit different in how it selects FirebaseApp instance since edit mode requires a separated instance but playmode will use DefaultInstance instead. (But the edit mode instance has AppOptions copied from DefaultInstance anyways)

Real device test

The ultimate test. After making all the tests available in Play Mode, you can click the button that says Run all in player (Android/iOS). The game will build now.

But in this build there are caveats :

"Oh no REST sucks, don't you know gRPC exist?"

In short I gave up, but it looks like a better than REST way if done right. It is just too messy with Unity. (In normal C# where NUGET is usable I would do RPC way.) Also the official C# API for Firestore uses the RPC + protobuf way, so no JSON mess like what I have here. One person even said he has successfully use gRPC from Unity, but since I have come a long way with UnityWebRequest I might as well continue using this as I wait. (But gRPC way will provide you with the interface most likely equal to upcoming Unity Firestore SDK, not an imitaion like Firestorm.)

What is it

It lets you do RPC with generated code, so it feels like you are calling regular function and it magically do remote calls. The C# side of code code is generated from Protobuf file from Google API repo.

What is the problem

Basically the "unloading assembly because it could cause crash in runtime" error message in Unity. Usually this message has something that cause the problem stated below but I have arrive at the point where nothing is stated after following all requirement by Nuget chain.

Some pointers if you want to try doing it gRPC way

First install gRPC stuff, there is a beta unitypackage by Google. See here for example you can see grpc_unity_package.1.19.0-dev.zip. Then install gRPC csharp plugin from somewhere, Google it and you should found it. It allows protoc to generate client stub methods when it see service syntax in the .proto file.

(At the time when you are reading this things might changed already.) Use artman with things in the googleapis repo. Go to here and follow it. You will be installing pipsi and starting docker daemon before you can use artman, then you will have to download Docker image of Google's artman by following the terminal. Note that all the things surrounding gRPC and googleapis seems to be sparsely documented than usual.

Finally you will be running something like :

~/.local/bin/artman --config google/firestore/artman_firestore.yaml generate csharp_gapic

The yaml file would be updated/changed in the future? I don't know..

You will now notice that the artman does not include the Firestore.Admin section, so you cannot do gRPC with admin API. Also it is missing some more references, you will have to install more Nuget package such as CommonProtos. And in an hours or two maybe you will arrive with the same "unloading assembly" error as me?

How about just installing Google.Cloud.Firestore and its dependencies

When I do

nuget install google.cloud.firestore -Prerelease

I got tons of related Nuget which in turn resolves into gRPC again. I think it is scary and difficult to get it working (at runtime too) so I didn't continue this path either. Let's trust UnityWebRequest!!

License

The license is MIT as you can see in LICENSE.txt, and to stress this is provided as-is without any warranty as said in the MIT license.

Blatant advertisement