Home

Awesome

ReTiSAR

Implementation of the Real-Time Spherical Array Renderer for binaural reproduction in Python [1][2].<br/> Mentioned in Awesome Python for Scientific Audio

Badge_OS Badge_Python Badge Version Badge_LastCommit <br/> Badge_CommitActivity Badge_CodeSize Badge_RepoSize Badge Black <br/> Badge_Conda [Badge_FFTW][FFTW] Badge_JACK [Badge_SOFA][SOFA] Badge_OSC

Contents: | Requirements | Setup | Quickstart | Execution parameters | Execution modes | Remote Control | Validation - Setup and Execution | Benchmark - Setup and Execution | References | Change Log | Contributing | Credits | License |

Requirements

Setup

Quickstart

JACK initialization — In case you have never started the JACK audio server on your system or want to make sure it initializes with appropriate values. Open the QjackCtl application to set your system specific default settings.<br/> At this point the only relevant JACK audio server setting is the sampling frequency, which has to match the sampling frequency of your rendered audio source file or stream (no resampling will be applied for that specific file).

FFTW optimization — In case the rendering takes very long to start (after the message "initializing FFTW DFT optimization ..."), you might want to endure this long computation time once (per rendering configuration) or lower your [FFTW] planner effort (see --help).

Rendering performance — Follow these remarks to expect continuous and artifact free rendering:

Always check the command line output or generated log files in case the rendering pipeline does not initialize successfully!

Execution parameters

The following parameters are all optional and available in combinations with the named execution modes subsequently:<br/>

Execution modes

This section list all the conceptually different rendering modes of the pipeline. Most of the other beforehand introduced execution parameters can be combined with the mode-specific parameters. In case no manual value for all specific rendering parameters is provided (as in the following examples), their respective default values will be used.

Most execution modes require additional external measurement data, which cannot be republished here. However, all provided examples are based on publicly available research data. Respective files are represented here by provided source reference files (see res/), containing a source URL and potentially further instructions. In case the respective resource data file is not yet available on your system, download instructions will be shown in the command line output and generated log files.

Note that the rendering performance is mostly determined by the chosen combination of the following parameters: number of microphones (ARIR channels), room reverberation time (ARIR length), IR truncation cutoff level and rendering block size.

Remote Control

Validation - Setup and Execution

Benchmark - Setup and Execution

References

[1] H. Helmholz, C. Andersson, and J. Ahrens, “Real-Time Implementation of Binaural Rendering of High-Order Spherical Microphone Array Signals,” in Fortschritte der Akustik -- DAGA 2019, 2019, pp. 1462–1465.<br/> [2] H. Helmholz, T. Lübeck, J. Ahrens, S. V. Amengual Garí, D. Lou Alon, and R. Mehra, “Updates on the Real-Time Spherical Array Renderer (ReTiSAR),” in Fortschritte der Akustik -- DAGA 2020, 2020, pp. 1169–1172.<br/> [3] B. Bernschütz, C. Pörschmann, S. Spors, and S. Weinzierl, “SOFiA Sound Field Analysis Toolbox,” in International Conference on Spatial Audio, 2011, pp. 7–15.<br/> [4] C. Hold, H. Gamper, V. Pulkki, N. Raghuvanshi, and I. J. Tashev, “Improving Binaural Ambisonics Decoding by Spherical Harmonics Domain Tapering and Coloration Compensation,” in International Conference on Acoustics, Speech and Signal Processing, 2019, pp. 261–265, doi: 10.1109/ICASSP.2019.8683751.<br/> [5] Z. Ben-Hur, F. Brinkmann, J. Sheaffer, S. Weinzierl, and B. Rafaely, “Spectral Equalization in Binaural Signals Represented by Order-Truncated Spherical Harmonics,” J. Acoust. Soc. Am., vol. 141, no. 6, pp. 4087–4096, 2017, doi: 10.1121/1.4983652.<br/> [6] B. Bernschütz, “A Spherical Far Field HRIR/HRTF Compilation of the Neumann KU 100,” in Fortschritte der Akustik -- AIA/DAGA 2013, 2013, pp. 592–595, doi: 10.5281/zenodo.3928296.<br/> [7] P. Majdak et al., “Spatially Oriented Format for Acoustics: A Data Exchange Format Representing Head-Related Transfer Functions,” in AES Convention 134, 2013, pp. 262–272.<br/> [8] C. Armstrong, L. Thresh, D. Murphy, and G. Kearney, “A Perceptual Evaluation of Individual and Non-Individual HRTFs: A Case Study of the SADIE II Database,” Appl. Sci., vol. 8, no. 11, pp. 1–21, 2018, doi: 10.3390/app8112029.<br/> [9] F. Brinkmann et al., “The FABIAN Head-Related Transfer Function Data Base.” Technische Universität Berlin, Berlin, Germany, 2017, doi: 10.14279/depositonce-5718.5.<br/> [10] H. Helmholz, D. Lou Alon, S. V. Amengual Garí, and J. Ahrens, “Instrumental Evaluation of Sensor Self-Noise in Binaural Rendering of Spherical Microphone Array Signals,” in Forum Acusticum, 2020, pp. 1349–1356, doi: 10.48465/fa.2020.0074.<br/> [11] P. Stade, B. Bernschütz, and M. Rühl, “A Spatial Audio Impulse Response Compilation Captured at the WDR Broadcast Studios,” in 27th Tonmeistertagung -- VDT International Convention, 2012, pp. 551–567.<br/> [12] C. Hohnerlein and J. Ahrens, “Spherical Microphone Array Processing in Python with the sound_field_analysis-py Toolbox,” in Fortschritte der Akustik -- DAGA 2017, 2017, pp. 1033–1036.<br/> [13] H. Helmholz, J. Ahrens, D. Lou Alon, S. V. Amengual Garí, and R. Mehra, “Evaluation of Sensor Self-Noise In Binaural Rendering of Spherical Microphone Array Signals,” in International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 161–165, doi: 10.1109/ICASSP40776.2020.9054434.<br/> [14] H. Helmholz, D. Lou Alon, S. V. Amengual Garí, and J. Ahrens, “Effects of Additive Noise in Binaural Rendering of Spherical Microphone Array Signals,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 3642–3653, 2021, doi: 10.1109/TASLP.2021.3129359.

Change Log

See CHANGE_LOG for full details.

Contributing

See CONTRIBUTING for full details.

Credits

Written by Hannes Helmholz.

Scientific supervision by Jens Ahrens.

Contributions by Carl Andersson and Tim Lübeck.

This work was funded by Facebook Reality Labs.

License

This software is licensed under a Non-Commercial Software License (see LICENSE for full details).

Copyright (c) 2018<br/> Division of Applied Acoustics<br/> Chalmers University of Technology

[SOFA]: https://www.sofaconventions.org/mediawiki/index.php/SOFA_(Spatially_Oriented_Format_for_Acoustics [FFTW]: http://www.fftw.org/fftw3_doc/Planner-Flags.html