Home

Awesome

VideoKit

Better Stack Badge

VideoKit is the only full feature user-generated content solution for Unity Engine. VideoKit allows:

Installing VideoKit

Add the following items to your Unity project's Packages/manifest.json:

{
  "scopedRegistries": [
    {
      "name": "VideoKit",
      "url": "https://registry.npmjs.com",
      "scopes": ["ai.videokit"]
    },
    {
      "name": "Function",
      "url": "https://registry.npmjs.com",
      "scopes": ["ai.fxn"]
    }
  ],
  "dependencies": {
    "ai.videokit.videokit": "0.0.22",
  }
}

[!IMPORTANT] VideoKit is still in alpha. As such, the C# API is expected to change a lot between releases.

Retrieving your Access Key

To use VideoKit, you will need to generate an access key. First, head over to videokit.ai to create an account by logging in. Once you do, generate an access key:

generating an access key

Then add the key to your Unity project in Project Settings > VideoKit:

set the access key

[!CAUTION] If your Unity project is open-source, make sure to add ProjectSettings/VideoKit.asset to your .gitignore file to keep your VideoKit access key private.

Using VideoKit

Here are a few things you can do with VideoKit:

Social Sharing

Share images, audio, and video files with the native share sheet with the MediaAsset.Share method:

Texture2D image = ...
ImageAsset asset = await MediaAsset.FromTexture(image);
string receiverAppId = await asset.Share();

Saving to the Camera Roll

Save images and videos to the camera roll with the MediaAsset.SaveToCameraRoll method:

Texture2D image = ...
ImageAsset asset = await MediaAsset.FromTexture(image);
bool saved = await asset.SaveToCameraRoll();

Picking from the Camera Roll

Pick images and videos from the camera roll with the MediaAsset.FromCameraRoll<T> method:

// This will present the native gallery UI
var asset = await MediaAsset.FromCameraRoll<ImageAsset>() as ImageAsset;
Texture2D image = await asset.ToTexture();
// Do stuff with `image`...

Camera Streaming

Stream the camera preview with the VideoKitCameraManager component:

stream the camera preview

Record Videos

Record MP4, HEVC, WEBM videos; animated GIF images; JPEG image sequences; and WAV audio files with the VideoKitRecorder component:

recording a video

Human Texture

Remove the background from the camera preview with the VideoKitCameraManager component:

using the human texture

Speech-to-Text

Caption audio with the AudioAsset.Caption method:

AudioClip clip = ...;
var asset = await MediaAsset.FromAudioClip(clip);
var caption = await asset.Caption();
Debug.Log(caption);

Text Commands

Convert a natural language prompt into a struct with the TextAsset.To<T> method. This enables features like text commands, and can be combined with audio captioning for voice control:

using System.ComponentModel; // for `DescriptionAttribute`
using VideoKit.Assets;

struct Command { // Define this however you want

    [Description(@"The user's name")]
    public string name;

    [Description(@"The user's age")]
    public int age;
}

async void ParseCommand () {
    var prompt = "My name is Jake and I'm thirteen years old.";
    var asset = await MediaAsset.FromText(prompt);
    var command = await asset.To<Command>();
    // command = { "name": "Jake", "age": 13 }
}

Requirements

Supported Platforms

Resources

Thank you very much!