Awesome
<div align="center"> <a href="https://vald.vdaas.org/"> <img src="./assets/image/readme.svg" width="50%" /> </a> </div>
What is Vald?
Vald is a highly scalable distributed fast approximate nearest neighbor (ANN) dense vector search engine.
Vald is designed and implemented based on Cloud-Native architecture.
Vald has automatic vector indexing and index backup, and horizontal scaling which made for searching from billions of feature vector data.
Vald is easy to use, feature-rich and highly customizable as you needed.
It uses the fastest ANN Algorithm NGT to search neighbors.
(If you are interested in ANN benchmarks, please refer to ann-benchmarks.com.)
For more information, please refer to Official Web Site.
<div align="center"> <img src="./assets/image/svg/vald_architecture_overview.svg" width="100%" /> </div>Vald can handle any object data, image, audio processing, video, text, binary, or etc., if converting to the vector, and be used for:
- Recognition
- Recommendation
- Detecting
- Grammar checker
- Real-time translator
- anything you want to do!
Requirements
- Kubernetes 1.19~
- AVX2 instructions (required by Vald Agent NGT)
Get Started
Go to Get Started page to try out Vald !
Installation
Using Helm
helm repo add vald https://vald.vdaas.org/charts
helm install vald-cluster vald/vald
If you use the default values.yaml, the nightly images will be installed.
Using Helm-operator
Please refer to vald-helm-operator.
Components
<table> <tr> <th>Component</th> <th>Docker image</th> <th>latest image</th> <th>nightly image</th> </tr> <tr> <td>Agent NGT</td> <td> <a href="https://hub.docker.com/r/vdaas/vald-agent-ngt"> <img src="https://img.shields.io/docker/pulls/vdaas/vald-agent-ngt?label=vdaas%2Fvald-agent-ngt&logo=docker&style=flat-square"/> </a><br/> <a href="https://github.com/orgs/vdaas/packages/container/package/vald/vald-agent-ngt"> <img src="https://img.shields.io/badge/ghcr.io-vdaas%2Fvald%2Fvald--agent--ngt-brightgreen?logo=docker&style=flat-square"/> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-agent-ngt/tags?page=1&name=latest"> <img src="https://img.shields.io/docker/v/vdaas/vald-agent-ngt/latest?label=vald-agent-ngt" /> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-agent-ngt/tags?page=1&name=nightly"> <img src="https://img.shields.io/docker/v/vdaas/vald-agent-ngt/nightly?label=vald-agent-ngt" /> </a> </td> </tr> <tr> <td>Agent Sidecar</td> <td> <a href="https://hub.docker.com/r/vdaas/vald-agent-sidecar"> <img src="https://img.shields.io/docker/pulls/vdaas/vald-agent-sidecar?label=vdaas%2Fvald-agent-sidecar&logo=docker&style=flat-square"/> </a><br/> <a href="https://github.com/orgs/vdaas/packages/container/package/vald/vald-agent-sidecar"> <img src="https://img.shields.io/badge/ghcr.io-vdaas%2Fvald%2Fvald--agent--sidecar-brightgreen?logo=docker&style=flat-square"/> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-agent-sidecar/tags?page=1&name=latest"> <img src="https://img.shields.io/docker/v/vdaas/vald-agent-sidecar/latest?label=vald-agent-sidecar" /> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-agent-sidecar/tags?page=1&name=nightly"> <img src="https://img.shields.io/docker/v/vdaas/vald-agent-sidecar/nightly?label=vald-agent-sidecar" /> </a> </td> </tr> <tr> <td>Discoverer</td> <td> <a href="https://hub.docker.com/r/vdaas/vald-discoverer-k8s"> <img src="https://img.shields.io/docker/pulls/vdaas/vald-discoverer-k8s?label=vdaas%2Fvald-discoverer-k8s&logo=docker&style=flat-square"/> </a><br/> <a href="https://github.com/orgs/vdaas/packages/container/package/vald/vald-discoverer-k8s"> <img src="https://img.shields.io/badge/ghcr.io-vdaas%2Fvald%2Fvald--discoverer--k8s-brightgreen?logo=docker&style=flat-square"/> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-discoverer-k8s/tags?page=1&name=latest"> <img src="https://img.shields.io/docker/v/vdaas/vald-discoverer-k8s/latest?label=vald-discoverer-k8s" /> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-discoverer-k8s/tags?page=1&name=nightly"> <img src="https://img.shields.io/docker/v/vdaas/vald-discoverer-k8s/nightly?label=vald-discoverer-k8s" /> </a> </td> </tr> <tr> <td>Gateways</td> <td> <a href="https://hub.docker.com/r/vdaas/vald-lb-gateway"> <img src="https://img.shields.io/docker/pulls/vdaas/vald-lb-gateway?label=vdaas%2Fvald-lb-gateway&logo=docker&style=flat-square"/> </a><br/> <a href="https://github.com/orgs/vdaas/packages/container/package/vald/vald-lb-gateway"> <img src="https://img.shields.io/badge/ghcr.io-vdaas%2Fvald%2Fvald--lb--gateway-brightgreen?logo=docker&style=flat-square"/> </a><br/> <a href="https://hub.docker.com/r/vdaas/vald-filter-gateway"> <img src="https://img.shields.io/docker/pulls/vdaas/vald-filter-gateway?label=vdaas%2Fvald-filter-gateway&logo=docker&style=flat-square"/> </a><br/> <a href="https://github.com/orgs/vdaas/packages/container/package/vald/vald-filter-gateway"> <img src="https://img.shields.io/badge/ghcr.io-vdaas%2Fvald%2Fvald--filter--gateway-brightgreen?logo=docker&style=flat-square"/> </a><br/> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-lb-gateway/tags?page=1&name=latest"> <img src="https://img.shields.io/docker/v/vdaas/vald-lb-gateway/latest?label=vald-lb-gateway" /> </a><br /> <a href="https://hub.docker.com/r/vdaas/vald-filter-gateway/tags?page=1&name=latest"> <img src="https://img.shields.io/docker/v/vdaas/vald-filter-gateway/latest?label=vald-filter-gateway" /> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-lb-gateway/tags?page=1&name=nightly"> <img src="https://img.shields.io/docker/v/vdaas/vald-lb-gateway/nightly?label=vald-lb-gateway" /> </a><br> <a href="https://hub.docker.com/r/vdaas/vald-filter-gateway/tags?page=1&name=nightly"> <img src="https://img.shields.io/docker/v/vdaas/vald-filter-gateway/nightly?label=vald-filter-gateway" /> </a><br /> </td> </tr> <tr> <td>Index Manager</td> <td> <a href="https://hub.docker.com/r/vdaas/vald-manager-index"> <img src="https://img.shields.io/docker/pulls/vdaas/vald-manager-index?label=vdaas%2Fvald-manager-index&logo=docker&style=flat-square"/> </a><br/> <a href="https://github.com/orgs/vdaas/packages/container/package/vald/vald-manager-index"> <img src="https://img.shields.io/badge/ghcr.io-vdaas%2Fvald%2Fvald--manager--index-brightgreen?logo=docker&style=flat-square"/> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-manager-index/tags?page=1&name=latest"> <img src="https://img.shields.io/docker/v/vdaas/vald-manager-index/latest?label=vald-index-manager" /> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-manager-index/tags?page=1&name=nightly"> <img src="https://img.shields.io/docker/v/vdaas/vald-manager-index/nightly?label=vald-index-manager" /> </a> </td> </tr> <tr> <td>Helm Operator</td> <td> <a href="https://hub.docker.com/r/vdaas/vald-helm-operator"> <img src="https://img.shields.io/docker/pulls/vdaas/vald-helm-operator?label=vdaas%2Fvald-helm-operator&logo=docker&style=flat-square"/> </a><br/> <a href="https://github.com/orgs/vdaas/packages/container/package/vald/vald-helm-operator"> <img src="https://img.shields.io/badge/ghcr.io-vdaas%2Fvald%2Fvald--helm--operator-brightgreen?logo=docker&style=flat-square"/> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-helm-operator/tags?page=1&name=latest"> <img src="https://img.shields.io/docker/v/vdaas/vald-helm-operator/latest?label=vald-helm-operator" /> </a> </td> <td> <a href="https://hub.docker.com/r/vdaas/vald-helm-operator/tags?page=1&name=nightly"> <img src="https://img.shields.io/docker/v/vdaas/vald-helm-operator/nightly?label=vald-helm-operator" /> </a> </td> </tr> </table>Docker images tagging policy:
nightly... latest build of main branchvX.X.X... released versionslatest... latest build of release versionsstable... latest long-term supported version
Tools
Vald Users
<p align="center"> <a href="https://www.lycorp.co.jp/en/" target="_blank"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./assets/image/vald-users/lycorp_white.png"> <source media="(prefers-color-scheme: light)" srcset="./assets/image/vald-users/lycorp_black.png"> <img alt="LY" src="./assets/image/vald-users/lycorp.png" width="150" height="120"> </picture> </a> <a href="https://jpsearch.go.jp/" target="_blank"> <img src="./assets/image/vald-users/japansearch_color.png" alt="jpsearch" width="150" height="120"/> </a> </p>Contribution
Please read the contribution guide.
Before your first commit to this repository, it is strongly recommended to run the commands below.
git clone https://github.com/vdaas/vald && cd vald
make init
Contributors
<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
Thanks goes to these wonderful people (emoji key):
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --> <!-- prettier-ignore-start --> <!-- markdownlint-disable --> <table> <tbody> <tr> <td align="center" valign="top" width="14.28%"><a href="https://kpango.com"><img src="https://avatars1.githubusercontent.com/u/9798091?v=4?s=100" width="100px;" alt="Yusuke Kato"/><br /><sub><b>Yusuke Kato</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=kpango" title="Code">💻</a> <a href="#design-kpango" title="Design">🎨</a> <a href="#maintenance-kpango" title="Maintenance">🚧</a> <a href="#projectManagement-kpango" title="Project Management">📆</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/rinx"><img src="https://avatars3.githubusercontent.com/u/1588935?v=4?s=100" width="100px;" alt="Rintaro Okamura"/><br /><sub><b>Rintaro Okamura</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=rinx" title="Code">💻</a> <a href="https://github.com/vdaas/vald/commits?author=rinx" title="Documentation">📖</a> <a href="#maintenance-rinx" title="Maintenance">🚧</a> <a href="#platform-rinx" title="Packaging/porting to new platform">📦</a></td> <td align="center" valign="top" width="14.28%"><a href="https://morimoto.dev/"><img src="https://avatars2.githubusercontent.com/u/413873?v=4?s=100" width="100px;" alt="Kosuke Morimoto"/><br /><sub><b>Kosuke Morimoto</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=kmrmt" title="Code">💻</a> <a href="#example-kmrmt" title="Examples">💡</a> <a href="#tool-kmrmt" title="Tools">🔧</a> <a href="https://github.com/vdaas/vald/commits?author=kmrmt" title="Tests">⚠️</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/vankichi"><img src="https://avatars3.githubusercontent.com/u/13959763?v=4?s=100" width="100px;" alt="Kiichiro YUKAWA"/><br /><sub><b>Kiichiro YUKAWA</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=vankichi" title="Documentation">📖</a> <a href="#maintenance-vankichi" title="Maintenance">🚧</a> <a href="https://github.com/vdaas/vald/commits?author=vankichi" title="Tests">⚠️</a> <a href="#tutorial-vankichi" title="Tutorials">✅</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/datelier"><img src="https://avatars3.githubusercontent.com/u/57349093?v=4?s=100" width="100px;" alt="datelier"/><br /><sub><b>datelier</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=datelier" title="Code">💻</a> <a href="#ideas-datelier" title="Ideas, Planning, & Feedback">🤔</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/kevindiu"><img src="https://avatars1.githubusercontent.com/u/1985382?v=4?s=100" width="100px;" alt="Kevin Diu"/><br /><sub><b>Kevin Diu</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=kevindiu" title="Documentation">📖</a> <a href="#example-kevindiu" title="Examples">💡</a> <a href="https://github.com/vdaas/vald/commits?author=kevindiu" title="Tests">⚠️</a> <a href="#tutorial-kevindiu" title="Tutorials">✅</a></td> <td align="center" valign="top" width="14.28%"><a href="https://twitter.com/hiroto_hlts2"><img src="https://avatars0.githubusercontent.com/u/25459661?v=4?s=100" width="100px;" alt="Hiroto Funakoshi"/><br /><sub><b>Hiroto Funakoshi</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=hlts2" title="Documentation">📖</a> <a href="#tool-hlts2" title="Tools">🔧</a> <a href="https://github.com/vdaas/vald/commits?author=hlts2" title="Tests">⚠️</a> <a href="#tutorial-hlts2" title="Tutorials">✅</a></td> </tr> <tr> <td align="center" valign="top" width="14.28%"><a href="https://github.com/taisuou"><img src="https://avatars0.githubusercontent.com/u/21119375?v=4?s=100" width="100px;" alt="taisho"/><br /><sub><b>taisho</b></sub></a><br /><a href="#design-taisuou" title="Design">🎨</a> <a href="https://github.com/vdaas/vald/commits?author=taisuou" title="Documentation">📖</a> <a href="#example-taisuou" title="Examples">💡</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/pgrimaud"><img src="https://avatars1.githubusercontent.com/u/1866496?v=4?s=100" width="100px;" alt="Pierre Grimaud"/><br /><sub><b>Pierre Grimaud</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=pgrimaud" title="Documentation">📖</a></td> <td align="center" valign="top" width="14.28%"><a href="https://omerkatz.com"><img src="https://avatars.githubusercontent.com/u/48936?v=4?s=100" width="100px;" alt="Omer Katz"/><br /><sub><b>Omer Katz</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=thedrow" title="Documentation">📖</a> <a href="#tutorial-thedrow" title="Tutorials">✅</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/zchee"><img src="https://avatars.githubusercontent.com/u/6366270?v=4?s=100" width="100px;" alt="Koichi Shiraishi"/><br /><sub><b>Koichi Shiraishi</b></sub></a><br /><a href="#a11y-zchee" title="Accessibility">️️️️♿️</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/liusy182"><img src="https://avatars.githubusercontent.com/u/3293332?v=4?s=100" width="100px;" alt="Siyuan Liu"/><br /><sub><b>Siyuan Liu</b></sub></a><br /><a href="#a11y-liusy182" title="Accessibility">️️️️♿️</a> <a href="#example-liusy182" title="Examples">💡</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/dotdc"><img src="https://avatars.githubusercontent.com/u/12827900?v=4?s=100" width="100px;" alt="David Calvert"/><br /><sub><b>David Calvert</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=dotdc" title="Documentation">📖</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/takuyaymd"><img src="https://avatars.githubusercontent.com/u/49614391?v=4?s=100" width="100px;" alt="takuyaymd"/><br /><sub><b>takuyaymd</b></sub></a><br /><a href="https://github.com/vdaas/vald/issues?q=author%3Atakuyaymd" title="Bug reports">🐛</a> <a href="https://github.com/vdaas/vald/commits?author=takuyaymd" title="Code">💻</a> <a href="#maintenance-takuyaymd" title="Maintenance">🚧</a></td> </tr> <tr> <td align="center" valign="top" width="14.28%"><a href="https://github.com/junsei-ando"><img src="https://avatars.githubusercontent.com/u/1892077?v=4?s=100" width="100px;" alt="junsei-ando"/><br /><sub><b>junsei-ando</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=junsei-ando" title="Documentation">📖</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/ykadowak"><img src="https://avatars.githubusercontent.com/u/60080334?v=4?s=100" width="100px;" alt="Yusuke Kadowaki"/><br /><sub><b>Yusuke Kadowaki</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=ykadowak" title="Code">💻</a> <a href="https://github.com/vdaas/vald/commits?author=ykadowak" title="Tests">⚠️</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/aknishid"><img src="https://avatars.githubusercontent.com/u/38955080?v=4?s=100" width="100px;" alt="aknishid"/><br /><sub><b>aknishid</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=aknishid" title="Code">💻</a> <a href="#maintenance-aknishid" title="Maintenance">🚧</a> <a href="https://github.com/vdaas/vald/commits?author=aknishid" title="Documentation">📖</a></td> <td align="center" valign="top" width="14.28%"><a href="https://hrichik.xyz"><img src="https://avatars.githubusercontent.com/u/57860916?v=4?s=100" width="100px;" alt="Hrichik Mazumder"/><br /><sub><b>Hrichik Mazumder</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=hrichiksite" title="Documentation">📖</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/smorihira"><img src="https://avatars.githubusercontent.com/u/105629359?v=4?s=100" width="100px;" alt="Shunya Morihira (森平 隼矢)"/><br /><sub><b>Shunya Morihira (森平 隼矢)</b></sub></a><br /><a href="#tool-smorihira" title="Tools">🔧</a> <a href="https://github.com/vdaas/vald/commits?author=smorihira" title="Code">💻</a></td> <td align="center" valign="top" width="14.28%"><a href="https://github.com/iammytoo"><img src="https://avatars.githubusercontent.com/u/64457274?v=4?s=100" width="100px;" alt="miyamoto"/><br /><sub><b>miyamoto</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=iammytoo" title="Code">💻</a> <a href="#research-iammytoo" title="Research">🔬</a></td> <td align="center" valign="top" width="14.28%"><a href="https://www.highpon.com/"><img src="https://avatars.githubusercontent.com/u/54130718?v=4?s=100" width="100px;" alt="s-shiraki"/><br /><sub><b>s-shiraki</b></sub></a><br /><a href="https://github.com/vdaas/vald/commits?author=highpon" title="Tests">⚠️</a> <a href="https://github.com/vdaas/vald/commits?author=highpon" title="Code">💻</a></td> </tr> </tbody> </table> <!-- markdownlint-restore --> <!-- prettier-ignore-end --> <!-- ALL-CONTRIBUTORS-LIST:END -->LICENSE
Vald released under Apache 2.0 license, refer LICENSE file.
