Awesome
FindSurface-visionOS
CurvSurf FindSurface™ library package for visionOS (Swift)
Overview
FindSurface-visionOS is a Swift package of FindSurface™ library, of which interface is modified to allow you to use its functionalities.
NOTE:
FindSurfaceFramework
(includingFindSurface-visionOS
) provides its functionality for non-commercial purposes within Apple Vision Pro devices. It is internally limited to process input point clouds of50k→ 100k points or less. For commercial uses or use cases that require more than50k→ 100k points, please contact to support@curvsurf.com.Starting from version 1.0.5, we have increased the size limit of the input point cloud from 50k to 100k. This adjustment addresses the increased number of vertices from
MeshAnchor
due to changes in visionOS 2.1, which were not officially mentioned by Apple (see details below.)
Samples
The followings are sample project repositories to help you get started to make your application with the package (more samples are to be added in the future):
- FindSurface-RealityKit-visionOS
- FindSurface-RealityKit-visionOS-Response-to-Request
- FindSurface-RealityKit-visionOS-Real-Time
- FindSurface-RealityKit-visionOS-Advertisement
To see how this package's APIs are called in each sample project, please refer to the following links:
- FindSurface-RealityKit-visionOS (Source file)
- FindSurface-RealityKit-visionOS-Response-to-Request (Source file).
- FindSurface-RealityKit-visionOS-Real-Time (Source code description)
How to install
Adding this package as a Dependency
You can import this package by adding the following line to the dependencies in your Package.swift
file:
dependencies: [
...
.package(utl: "https://github.com/CurvSurf/FindSurface-visionOS", from: "1.0.0")
],
targets: [
.target(name: "<target>", dependencies: [
...
"FindSurface-visionOS"
]),
...
]
Then, add import FindSurface_visionOS
(note that it is not hyphenated, but underscored in the middle of the words.)
Using the XCFramework without the wrapper
This package is a wrapper containing the visionOS version (including the simulator) of the FindSurface XCFramework. You can also refer to the source code of this package to use the framework directly. In that case, import FindSurfaceFramework
instead.
Optimal Parameter Set for Apple Vision Pro
NOTE: Due to the limited accuracy of
MeshAnchor
, which is generated by Apple Vision Pro, the size of the objects that can be detected or measured usingFindSurface
is also practically limited in a range from 1 meter to 20 meters.
Each parameter of FindSurface provides essential information that helps FindSurface detect and measure object surfaces from the given points. For detailed explanations of each parameter, please refer to FindSurface.
Once measurement points are collected, measurement accuracy
and mean distance
should be determined as specific values corresponding to the collected points. This is because these parameters reflect the characteristics of the points.
FindSurface works based on these values and requires additional parameters such as the seed radius
, which is proportional to the approximate diameter of objects to be detected/measured, and lateral extension
/radial expansion
for region growing, depending on the lengths/widths of the objects if necessary.
NOTE: With the recent visionOS 2.1 update, we noticed significant changes in the data provided by the
MeshAnchor
on Apple Vision Pro. (The following observations are based on our own analysis through our app and are not officially mentioned in Apple's patch notes.)
- The coverage of individual
MeshAnchor
has decreased, while the number of anchors has increased.- The quad sizes in the mesh data (formed by two adjacent triangles with shared edges) have changed. Previously, a mix of 12 x 12 cm and 6 x 6 cm quads were commonly observed. After the update, most quads are now 6 x 6 cm, and 12 x 12 cm quads seem to no longer appear. It means the number of vertices has increased for the same coverage.
These changes result in an increase in the resolution of the
MeshAnchor
data with a corresponding decrease in the smoothing effect due to mesh generation. While this allows for more detailed representation of the convex/concave characteristics of real-world geometry, the orthogonal distance error of mesh vertices to the actual geometry surface tends to become more variable (i.e., increased points with errors).Since the characteristics of the mesh data provided by Apple Vision Pro have changed, the optimal parameter values for FindSurface must also be updated.
Our sample apps' source code reflects these parameter changes as the new defaults. However, since these parameters are overwritten by user-defined values stored via
UserDefaults
, they need to be manually adjusted by the user in previously installed app.
Based on our analysis, we recommend the following adjustments to the optimal set of parameters:
- Accuracy:
1.5 cm→ 2.0 cm ~ 2.5 cm - Mean Distance:
10 cm→ 5 cm - Seed Radius: 1/10 ~ 1/2 of object's diameter/width in centimeter (10 cm as default and 5 cm as minimum)
- Lat. Ext.:
10→ off - Rad. Exp.: 5
As an Apple Vision Pro user, you only need to set the seed radius proportional to the size of the object using the above parameter presets as default. For planes having small areas or short cylinders/cones, it is recommended to set the lateral extension to below 5
. To detect or measure an object smaller than 1 meter (although we don't recommend it), set the radial expansion to 4
or less.
About License
You may use the source code of this package freely under MIT license, as the license file stated, except for FindSurfaceFramework
, which is the core of the package.
About the framework, refer to the following statement:
Copyright (c) 2024 CurvSurf, Inc. All rights reserved.
The framework `FindSurface-visionOS.xcframework` and `FindSurfaceFramework.framework`'s
ownership is solely on CurvSurf, Inc. and anyone can use it for non-commercial purposes.
Contact to support@curvsurf.com for commercial use of the library.