Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(audio): real noise suppression via vendored RNNoise (send + receive)
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Details
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Details

Browse Source 
The two-sided NR plumbing (RemoteStream::recv_ns + the per-listener
vc_set_remote_stream noise_reduction toggle) was wired but inert:
ApmProcessor::create() returned a no-op passthrough, because the
originally-planned webrtc-audio-processing has no working Windows/macOS
build. Drop in RNNoise as the real backend behind the same ApmProcessor
interface, lighting up both NR paths.

- Vendor RNNoise (BSD-3 + CC0) at third_party/rnnoise/ — the vcpkg port
  is !windows !arm, so it can't cover our primary targets. Shrunk int8
  model (78MB -> 11.7MB via upstream scripts/shrink_model.sh), built as a
  standalone C static lib with no RTCD (portable scalar path on x86,
  auto-NEON on arm64) under -DDISABLE_DEBUG_FLOAT. Model is baked in
  (rnnoise_create(NULL)); no runtime file.
- New RnnoiseProcessor (core/src/audio/apm_processor.cpp) selected by
  ApmProcessor::create() when VOICECAT_HAS_NS. Mono/48kHz/480-sample;
  our clock is fixed 48kHz and Opus frame sizes are multiples of 480, so
  no resampling. RT-safe: allocates at construction, lock-free in the
  capture/playback callbacks.
- Receive-side: lit up via the factory; gated to mono streams (a stereo
  stream is a screen-audio share, not voice).
- Send-side (new): vc_set_input_noise_reduction(client, enable) ABI +
  vc_client::mic_ns_, run before input gain/VAD in on_capture_frame. A
  stereo mic is downmixed to mono ONLY when NR is on — with NR off a
  stereo mic keeps full stereo (never collapse mic quality unasked).
- Enable C as a project language for the vendored lib.
- New noise_suppression test: white noise through ApmProcessor::create()
  drops ~99.9% RMS. ctest --preset dev green, 28/28. windows-client DLL
  builds clean with vc_set_input_noise_reduction exported, system-only deps.
- Docs synced: voice.md §10, tech-stack.md §1/§5, third_party/README.md,
  vcpkg.json note, PROGRESS.md, CLAUDE.md.

Client on/off UI toggles (Windows/macOS/iOS) are the remaining follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-23 13:30:54 +02:00
parent 7249a8fd30
commit bad9c7533a
50 changed files with 381395 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
third_party/rnnoise/AUTHORS vendored Normal file
View File

@@ -0,0 +1 @@
Jean-Marc Valin <jmvalin@jmvalin.ca>

32
third_party/rnnoise/COPYING vendored Normal file
View File

@@ -0,0 +1,32 @@
Copyright (c) 2007-2017, 2024 Jean-Marc Valin
Copyright (c) 2023 Amazon
Copyright (c) 2017, Mozilla
Copyright (c) 2005-2017, Xiph.Org Foundation
Copyright (c) 2003-2004, Mark Borgerding
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
- Neither the name of the Xiph.Org Foundation nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

125
third_party/rnnoise/README vendored Normal file
View File

@@ -0,0 +1,125 @@
RNNoise is a noise suppression library based on a recurrent neural network.
A description of the algorithm is provided in the following paper:
J.-M. Valin, A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech
Enhancement, Proceedings of IEEE Multimedia Signal Processing (MMSP) Workshop,
arXiv:1709.08243, 2018.
https://arxiv.org/pdf/1709.08243.pdf
An interactive demo of version 0.1 is available at: https://jmvalin.ca/demo/rnnoise/
To compile, just type:
% ./autogen.sh
% ./configure
% make
Optionally:
% make install
It is recommended to either set -march= in the CFLAGS to an architecture
with AVX2 support or to add --enable-x86-rtcd to the configure script
so that AVX2 (or SSE4.1) can at least be used as an option.
Note that the autogen.sh script will automatically download the model files
from the Xiph.Org servers, since those are too large to put in Git.
While it is meant to be used as a library, a simple command-line tool is
provided as an example. It operates on RAW 16-bit (machine endian) mono
PCM files sampled at 48 kHz. It can be used as:
% ./examples/rnnoise_demo <noisy speech> <output denoised>
The output is also a 16-bit raw PCM file.
NOTE AGAIN, THE INPUT and OUTPUT ARE IN RAW FORMAT, NOT WAV.
The latest version of the source is available from
https://gitlab.xiph.org/xiph/rnnoise . The GitHub repository
is a convenience copy.
== Training ==
The models distributed with RNNoise are now trained using only the publicly
available datasets listed below and using the training precedure described
here. Exact results will still depend on the the exact mix of data used,
on how long the training is performed and on the various random seeds involved.
To train an RNNoise model, you need both clean speech data, and noise data.
Both need to be sampled at 48 kHz, in 16-bit PCM format (machine endian).
Clean speech data can be obtained from the datasets listed in the datasets.txt
file, or by downloaded the already-concatenation of those files in
https://media.xiph.org/rnnoise/data/tts_speech_48k.sw
For noise data, we suggest the background_noise.sw and foreground_noise.sw
(or later versions) noise files from https://media.xiph.org/rnnoise/data/
The foreground_noise.sw file contains noise signals that are meant to be added
to the background noise (e.g. keyboard sounds). Optionally, the foreground noise
file can even be denoised with a traditional denoiser (e.g. libspeexdsp) to
keep only the transient components. For background noise, the data from the
original RNNoise noise collection have now been sufficiently filtered to
provide good results -- either alone or in combination with the
background_noise.sw file. The dataset can be downloaded (updated Jan 30th 2025)
from: https://media.xiph.org/rnnoise/rnnoise_contributions.tar.gz
The first step is to take the speech and noise, and mix them in a variety of
ways to simulate real life conditions (including pauses, filtering and more).
Assuming the files are called speech.pcm and noise.pcm, start by generating
the training feature data with:
% ./dump_features speech.pcm background_noise.pcm foreground_noise.pcm features.f32 <count>
where <count> is the number of sequences to process. The number of sequences
should be at least 10000, but the more the better (200000 or more is
recommended).
Optionally, training can also simulate reverberation, in which case room impulse
responses (RIR) are also needed. Limited RIR data is available at:
https://media.xiph.org/rnnoise/data/measured_rirs-v2.tar.gz
The format for those is raw 32-bit floating-point (files are little endian).
Assuming a list of all the RIR files is contained in a rir_list.txt file,
the training feature data can be generated with:
% ./dump_features -rir_list rir_list.txt speech.pcm background_noise.pcm foreground_noise.pcm features.f32 <count>
To make the feature generation faster, you can use the script provided in
script/dump_features_parallel.sh (you will need to modify the script if you
want to add RIR augmentation).
To use it:
% script/dump_features_parallel.sh ./dump_features speech.pcm background_noise.pcm foreground_noise.pcm features.f32 <count> rir_list.txt
which will run nb_processes processes, each for count sequences, and
concatenate the output to a single file.
Once the feature file is computed, you can start the training with:
% python3 train_rnnoise.py features.f32 output_directory
Choose a number of epochs (using --epochs) that leads to about 75000 weight
updates. The training will produce .pth files, e.g. rnnoise_50.pth .
The next step is to convert the model to C files using:
% python3 dump_rnnoise_weights.py --quantize rnnoise_50.pth rnnoise_c
which will produce the rnnoise_data.c and rnnoise_data.h files in the
rnnoise_c directory.
Copy these files to src/ and then build RNNoise using the instructions above.
For slightly better results, a trained model can be used to remove any noise
from the "clean" training speech, before restaring the denoising process
again (no need to do that more than once).
== Loadable Models ==
The model format has changed since v0.1.1. Models now use a binary
"machine endian" format. To output a model in that format, build RNNoise
with that model and use the dump_weights_blob executable to output a
weights_blob.bin binary file. That file can then be used with the
rnnoise_model_from_file() API call. Note that the model object MUST NOT
be deleted while the RNNoise state is active and the file MUST NOT
be closed.
To avoid including the default model in the build (e.g. to reduce download
size) and rely only on model loading, add -DUSE_WEIGHTS_FILE to the CFLAGS.
To be able to load different models, the model size (and header file) needs
to patch the size use during build. Otherwise the model will not load
We provide a "little" model with half as an alternative. To use the smaller
model, rename rnnoise_data_little.c to rnnoise_data.c. It is possible
to build both the regular and little binary weights and load any of them
at run time since the little model has the same size as the regular one
(except for the increased sparsity).

131
third_party/rnnoise/include/rnnoise.h vendored Normal file
View File

@@ -0,0 +1,131 @@
/* Copyright (c) 2018 Gregor Richards
* Copyright (c) 2017 Mozilla */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef RNNOISE_H
#define RNNOISE_H 1
#include <stdio.h>
#ifdef __cplusplus
extern "C" {
#endif
#ifndef RNNOISE_EXPORT
# if defined(WIN32)
# if defined(RNNOISE_BUILD) && defined(DLL_EXPORT)
# define RNNOISE_EXPORT __declspec(dllexport)
# else
# define RNNOISE_EXPORT
# endif
# elif defined(__GNUC__) && defined(RNNOISE_BUILD)
# define RNNOISE_EXPORT __attribute__ ((visibility ("default")))
# else
# define RNNOISE_EXPORT
# endif
#endif
typedef struct DenoiseState DenoiseState;
typedef struct RNNModel RNNModel;
/**
* Return the size of DenoiseState
*/
RNNOISE_EXPORT int rnnoise_get_size(void);
/**
* Return the number of samples processed by rnnoise_process_frame at a time
*/
RNNOISE_EXPORT int rnnoise_get_frame_size(void);
/**
* Initializes a pre-allocated DenoiseState
*
* If model is NULL the default model is used.
*
* See: rnnoise_create() and rnnoise_model_from_file()
*/
RNNOISE_EXPORT int rnnoise_init(DenoiseState *st, RNNModel *model);
/**
* Allocate and initialize a DenoiseState
*
* If model is NULL the default model is used.
*
* The returned pointer MUST be freed with rnnoise_destroy().
*/
RNNOISE_EXPORT DenoiseState *rnnoise_create(RNNModel *model);
/**
* Free a DenoiseState produced by rnnoise_create.
*
* The optional custom model must be freed by rnnoise_model_free() after.
*/
RNNOISE_EXPORT void rnnoise_destroy(DenoiseState *st);
/**
* Denoise a frame of samples
*
* in and out must be at least rnnoise_get_frame_size() large.
*/
RNNOISE_EXPORT float rnnoise_process_frame(DenoiseState *st, float *out, const float *in);
/**
* Load a model from a memory buffer
*
* It must be deallocated with rnnoise_model_free() and the buffer must remain
* valid until after the returned object is destroyed.
*/
RNNOISE_EXPORT RNNModel *rnnoise_model_from_buffer(const void *ptr, int len);
/**
* Load a model from a file
*
* It must be deallocated with rnnoise_model_free() and the file must not be
* closed until the returned object is destroyed.
*/
RNNOISE_EXPORT RNNModel *rnnoise_model_from_file(FILE *f);
/**
* Load a model from a file name
*
* It must be deallocated with rnnoise_model_free()
*/
RNNOISE_EXPORT RNNModel *rnnoise_model_from_filename(const char *filename);
/**
* Free a custom model
*
* It must be called after all the DenoiseStates referring to it are freed.
*/
RNNOISE_EXPORT void rnnoise_model_free(RNNModel *model);
#ifdef __cplusplus
}
#endif
#endif

182
third_party/rnnoise/src/_kiss_fft_guts.h vendored Normal file
View File

@@ -0,0 +1,182 @@
/*Copyright (c) 2003-2004, Mark Borgerding
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.*/
#ifndef KISS_FFT_GUTS_H
#define KISS_FFT_GUTS_H
#define MIN(a,b) ((a)<(b) ? (a):(b))
#define MAX(a,b) ((a)>(b) ? (a):(b))
/* kiss_fft.h
defines kiss_fft_scalar as either short or a float type
and defines
typedef struct { kiss_fft_scalar r; kiss_fft_scalar i; }kiss_fft_cpx; */
#include "kiss_fft.h"
/*
Explanation of macros dealing with complex math:
C_MUL(m,a,b) : m = a*b
C_FIXDIV( c , div ) : if a fixed point impl., c /= div. noop otherwise
C_SUB( res, a,b) : res = a - b
C_SUBFROM( res , a) : res -= a
C_ADDTO( res , a) : res += a
* */
#ifdef FIXED_POINT
#include "arch.h"
#define SAMP_MAX 2147483647
#define TWID_MAX 32767
#define TRIG_UPSCALE 1
#define SAMP_MIN -SAMP_MAX
# define S_MUL(a,b) MULT16_32_Q15(b, a)
# define C_MUL(m,a,b) \
do{ (m).r = SUB32_ovflw(S_MUL((a).r,(b).r) , S_MUL((a).i,(b).i)); \
(m).i = ADD32_ovflw(S_MUL((a).r,(b).i) , S_MUL((a).i,(b).r)); }while(0)
# define C_MULC(m,a,b) \
do{ (m).r = ADD32_ovflw(S_MUL((a).r,(b).r) , S_MUL((a).i,(b).i)); \
(m).i = SUB32_ovflw(S_MUL((a).i,(b).r) , S_MUL((a).r,(b).i)); }while(0)
# define C_MULBYSCALAR( c, s ) \
do{ (c).r = S_MUL( (c).r , s ) ;\
(c).i = S_MUL( (c).i , s ) ; }while(0)
# define DIVSCALAR(x,k) \
(x) = S_MUL( x, (TWID_MAX-((k)>>1))/(k)+1 )
# define C_FIXDIV(c,div) \
do { DIVSCALAR( (c).r , div); \
DIVSCALAR( (c).i , div); }while (0)
#define C_ADD( res, a,b)\
do {(res).r=ADD32_ovflw((a).r,(b).r); (res).i=ADD32_ovflw((a).i,(b).i); \
}while(0)
#define C_SUB( res, a,b)\
do {(res).r=SUB32_ovflw((a).r,(b).r); (res).i=SUB32_ovflw((a).i,(b).i); \
}while(0)
#define C_ADDTO( res , a)\
do {(res).r = ADD32_ovflw((res).r, (a).r); (res).i = ADD32_ovflw((res).i,(a).i);\
}while(0)
#define C_SUBFROM( res , a)\
do {(res).r = ADD32_ovflw((res).r,(a).r); (res).i = SUB32_ovflw((res).i,(a).i); \
}while(0)
#if defined(OPUS_ARM_INLINE_ASM)
#include "arm/kiss_fft_armv4.h"
#endif
#if defined(OPUS_ARM_INLINE_EDSP)
#include "arm/kiss_fft_armv5e.h"
#endif
#if defined(MIPSr1_ASM)
#include "mips/kiss_fft_mipsr1.h"
#endif
#else /* not FIXED_POINT*/
# define S_MUL(a,b) ( (a)*(b) )
#define C_MUL(m,a,b) \
do{ (m).r = (a).r*(b).r - (a).i*(b).i;\
(m).i = (a).r*(b).i + (a).i*(b).r; }while(0)
#define C_MULC(m,a,b) \
do{ (m).r = (a).r*(b).r + (a).i*(b).i;\
(m).i = (a).i*(b).r - (a).r*(b).i; }while(0)
#define C_MUL4(m,a,b) C_MUL(m,a,b)
# define C_FIXDIV(c,div) /* NOOP */
# define C_MULBYSCALAR( c, s ) \
do{ (c).r *= (s);\
(c).i *= (s); }while(0)
#endif
#ifndef CHECK_OVERFLOW_OP
# define CHECK_OVERFLOW_OP(a,op,b) /* noop */
#endif
#ifndef C_ADD
#define C_ADD( res, a,b)\
do { \
CHECK_OVERFLOW_OP((a).r,+,(b).r)\
CHECK_OVERFLOW_OP((a).i,+,(b).i)\
(res).r=(a).r+(b).r; (res).i=(a).i+(b).i; \
}while(0)
#define C_SUB( res, a,b)\
do { \
CHECK_OVERFLOW_OP((a).r,-,(b).r)\
CHECK_OVERFLOW_OP((a).i,-,(b).i)\
(res).r=(a).r-(b).r; (res).i=(a).i-(b).i; \
}while(0)
#define C_ADDTO( res , a)\
do { \
CHECK_OVERFLOW_OP((res).r,+,(a).r)\
CHECK_OVERFLOW_OP((res).i,+,(a).i)\
(res).r += (a).r; (res).i += (a).i;\
}while(0)
#define C_SUBFROM( res , a)\
do {\
CHECK_OVERFLOW_OP((res).r,-,(a).r)\
CHECK_OVERFLOW_OP((res).i,-,(a).i)\
(res).r -= (a).r; (res).i -= (a).i; \
}while(0)
#endif /* C_ADD defined */
#ifdef FIXED_POINT
/*# define KISS_FFT_COS(phase) TRIG_UPSCALE*floor(MIN(32767,MAX(-32767,.5+32768 * cos (phase))))
# define KISS_FFT_SIN(phase) TRIG_UPSCALE*floor(MIN(32767,MAX(-32767,.5+32768 * sin (phase))))*/
# define KISS_FFT_COS(phase) floor(.5+TWID_MAX*cos (phase))
# define KISS_FFT_SIN(phase) floor(.5+TWID_MAX*sin (phase))
# define HALF_OF(x) ((x)>>1)
#elif defined(USE_SIMD)
# define KISS_FFT_COS(phase) _mm_set1_ps( cos(phase) )
# define KISS_FFT_SIN(phase) _mm_set1_ps( sin(phase) )
# define HALF_OF(x) ((x)*_mm_set1_ps(.5f))
#else
# define KISS_FFT_COS(phase) (kiss_fft_scalar) cos(phase)
# define KISS_FFT_SIN(phase) (kiss_fft_scalar) sin(phase)
# define HALF_OF(x) ((x)*.5f)
#endif
#define kf_cexp(x,phase) \
do{ \
(x)->r = KISS_FFT_COS(phase);\
(x)->i = KISS_FFT_SIN(phase);\
}while(0)
#define kf_cexp2(x,phase) \
do{ \
(x)->r = TRIG_UPSCALE*celt_cos_norm((phase));\
(x)->i = TRIG_UPSCALE*celt_cos_norm((phase)-32768);\
}while(0)
#endif /* KISS_FFT_GUTS_H */

261
third_party/rnnoise/src/arch.h vendored Normal file
View File

@@ -0,0 +1,261 @@
/* Copyright (c) 2003-2008 Jean-Marc Valin
Copyright (c) 2007-2008 CSIRO
Copyright (c) 2007-2009 Xiph.Org Foundation
Written by Jean-Marc Valin */
/**
@file arch.h
@brief Various architecture definitions for CELT
*/
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef ARCH_H
#define ARCH_H
#include "opus_types.h"
#include "common.h"
# if !defined(__GNUC_PREREQ)
# if defined(__GNUC__)&&defined(__GNUC_MINOR__)
# define __GNUC_PREREQ(_maj,_min) \
((__GNUC__<<16)+__GNUC_MINOR__>=((_maj)<<16)+(_min))
# else
# define __GNUC_PREREQ(_maj,_min) 0
# endif
# endif
#define CELT_SIG_SCALE 32768.f
#define celt_fatal(str) _celt_fatal(str, __FILE__, __LINE__);
#ifdef ENABLE_ASSERTIONS
#include <stdio.h>
#include <stdlib.h>
#ifdef __GNUC__
__attribute__((noreturn))
#endif
static OPUS_INLINE void _celt_fatal(const char *str, const char *file, int line)
{
fprintf (stderr, "Fatal (internal) error in %s, line %d: %s\n", file, line, str);
abort();
}
#define celt_assert(cond) {if (!(cond)) {celt_fatal("assertion failed: " #cond);}}
#define celt_assert2(cond, message) {if (!(cond)) {celt_fatal("assertion failed: " #cond "\n" message);}}
#else
#define celt_assert(cond)
#define celt_assert2(cond, message)
#endif
#define IMUL32(a,b) ((a)*(b))
#define MIN16(a,b) ((a) < (b) ? (a) : (b)) /**< Minimum 16-bit value. */
#define MAX16(a,b) ((a) > (b) ? (a) : (b)) /**< Maximum 16-bit value. */
#define MIN32(a,b) ((a) < (b) ? (a) : (b)) /**< Minimum 32-bit value. */
#define MAX32(a,b) ((a) > (b) ? (a) : (b)) /**< Maximum 32-bit value. */
#define IMIN(a,b) ((a) < (b) ? (a) : (b)) /**< Minimum int value. */
#define IMAX(a,b) ((a) > (b) ? (a) : (b)) /**< Maximum int value. */
#define UADD32(a,b) ((a)+(b))
#define USUB32(a,b) ((a)-(b))
/* Set this if opus_int64 is a native type of the CPU. */
/* Assume that all LP64 architectures have fast 64-bit types; also x86_64
(which can be ILP32 for x32) and Win64 (which is LLP64). */
#if defined(__x86_64__) || defined(__LP64__) || defined(_WIN64)
#define OPUS_FAST_INT64 1
#else
#define OPUS_FAST_INT64 0
#endif
#define PRINT_MIPS(file)
#ifdef FIXED_POINT
typedef opus_int16 opus_val16;
typedef opus_int32 opus_val32;
typedef opus_int64 opus_val64;
typedef opus_val32 celt_sig;
typedef opus_val16 celt_norm;
typedef opus_val32 celt_ener;
#define Q15ONE 32767
#define SIG_SHIFT 12
/* Safe saturation value for 32-bit signals. Should be less than
2^31*(1-0.85) to avoid blowing up on DC at deemphasis.*/
#define SIG_SAT (300000000)
#define NORM_SCALING 16384
#define DB_SHIFT 10
#define EPSILON 1
#define VERY_SMALL 0
#define VERY_LARGE16 ((opus_val16)32767)
#define Q15_ONE ((opus_val16)32767)
#define SCALEIN(a) (a)
#define SCALEOUT(a) (a)
#define ABS16(x) ((x) < 0 ? (-(x)) : (x))
#define ABS32(x) ((x) < 0 ? (-(x)) : (x))
static OPUS_INLINE opus_int16 SAT16(opus_int32 x) {
return x > 32767 ? 32767 : x < -32768 ? -32768 : (opus_int16)x;
}
#ifdef FIXED_DEBUG
#include "fixed_debug.h"
#else
#include "fixed_generic.h"
#ifdef OPUS_ARM_PRESUME_AARCH64_NEON_INTR
#include "arm/fixed_arm64.h"
#elif OPUS_ARM_INLINE_EDSP
#include "arm/fixed_armv5e.h"
#elif defined (OPUS_ARM_INLINE_ASM)
#include "arm/fixed_armv4.h"
#elif defined (BFIN_ASM)
#include "fixed_bfin.h"
#elif defined (TI_C5X_ASM)
#include "fixed_c5x.h"
#elif defined (TI_C6X_ASM)
#include "fixed_c6x.h"
#endif
#endif
#else /* FIXED_POINT */
typedef float opus_val16;
typedef float opus_val32;
typedef float opus_val64;
typedef float celt_sig;
typedef float celt_norm;
typedef float celt_ener;
#ifdef FLOAT_APPROX
/* This code should reliably detect NaN/inf even when -ffast-math is used.
Assumes IEEE 754 format. */
static OPUS_INLINE int celt_isnan(float x)
{
union {float f; opus_uint32 i;} in;
in.f = x;
return ((in.i>>23)&0xFF)==0xFF && (in.i&0x007FFFFF)!=0;
}
#else
#ifdef __FAST_MATH__
#error Cannot build libopus with -ffast-math unless FLOAT_APPROX is defined. This could result in crashes on extreme (e.g. NaN) input
#endif
#define celt_isnan(x) ((x)!=(x))
#endif
#define Q15ONE 1.0f
#define NORM_SCALING 1.f
#define EPSILON 1e-15f
#define VERY_SMALL 1e-30f
#define VERY_LARGE16 1e15f
#define Q15_ONE ((opus_val16)1.f)
/* This appears to be the same speed as C99's fabsf() but it's more portable. */
#define ABS16(x) ((float)fabs(x))
#define ABS32(x) ((float)fabs(x))
#define QCONST16(x,bits) (x)
#define QCONST32(x,bits) (x)
#define NEG16(x) (-(x))
#define NEG32(x) (-(x))
#define NEG32_ovflw(x) (-(x))
#define EXTRACT16(x) (x)
#define EXTEND32(x) (x)
#define SHR16(a,shift) (a)
#define SHL16(a,shift) (a)
#define SHR32(a,shift) (a)
#define SHL32(a,shift) (a)
#define PSHR32(a,shift) (a)
#define VSHR32(a,shift) (a)
#define PSHR(a,shift) (a)
#define SHR(a,shift) (a)
#define SHL(a,shift) (a)
#define SATURATE(x,a) (x)
#define SATURATE16(x) (x)
#define ROUND16(a,shift) (a)
#define SROUND16(a,shift) (a)
#define HALF16(x) (.5f*(x))
#define HALF32(x) (.5f*(x))
#define ADD16(a,b) ((a)+(b))
#define SUB16(a,b) ((a)-(b))
#define ADD32(a,b) ((a)+(b))
#define SUB32(a,b) ((a)-(b))
#define ADD32_ovflw(a,b) ((a)+(b))
#define SUB32_ovflw(a,b) ((a)-(b))
#define MULT16_16_16(a,b) ((a)*(b))
#define MULT16_16(a,b) ((opus_val32)(a)*(opus_val32)(b))
#define MAC16_16(c,a,b) ((c)+(opus_val32)(a)*(opus_val32)(b))
#define MULT16_32_Q15(a,b) ((a)*(b))
#define MULT16_32_Q16(a,b) ((a)*(b))
#define MULT32_32_Q31(a,b) ((a)*(b))
#define MAC16_32_Q15(c,a,b) ((c)+(a)*(b))
#define MAC16_32_Q16(c,a,b) ((c)+(a)*(b))
#define MULT16_16_Q11_32(a,b) ((a)*(b))
#define MULT16_16_Q11(a,b) ((a)*(b))
#define MULT16_16_Q13(a,b) ((a)*(b))
#define MULT16_16_Q14(a,b) ((a)*(b))
#define MULT16_16_Q15(a,b) ((a)*(b))
#define MULT16_16_P15(a,b) ((a)*(b))
#define MULT16_16_P13(a,b) ((a)*(b))
#define MULT16_16_P14(a,b) ((a)*(b))
#define MULT16_32_P16(a,b) ((a)*(b))
#define DIV32_16(a,b) (((opus_val32)(a))/(opus_val16)(b))
#define DIV32(a,b) (((opus_val32)(a))/(opus_val32)(b))
#define SCALEIN(a) ((a)*CELT_SIG_SCALE)
#define SCALEOUT(a) ((a)*(1/CELT_SIG_SCALE))
#define SIG2WORD16(x) (x)
#endif /* !FIXED_POINT */
#ifndef GLOBAL_STACK_SIZE
#ifdef FIXED_POINT
#define GLOBAL_STACK_SIZE 120000
#else
#define GLOBAL_STACK_SIZE 120000
#endif
#endif
#endif /* ARCH_H */

174
third_party/rnnoise/src/celt_lpc.c vendored Normal file
View File

@@ -0,0 +1,174 @@
/* Copyright (c) 2009-2010 Xiph.Org Foundation
Written by Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "celt_lpc.h"
#include "arch.h"
#include "common.h"
#include "pitch.h"
#include "denoise.h"
void rnn_lpc(
opus_val16 *_lpc, /* out: [0...p-1] LPC coefficients */
const opus_val32 *ac, /* in: [0...p] autocorrelation values */
int p
)
{
int i, j;
opus_val32 r;
opus_val32 error = ac[0];
#ifdef FIXED_POINT
opus_val32 lpc[LPC_ORDER];
#else
float *lpc = _lpc;
#endif
RNN_CLEAR(lpc, p);
if (ac[0] != 0)
{
for (i = 0; i < p; i++) {
/* Sum up this iteration's reflection coefficient */
opus_val32 rr = 0;
for (j = 0; j < i; j++)
rr += MULT32_32_Q31(lpc[j],ac[i - j]);
rr += SHR32(ac[i + 1],3);
r = -SHL32(rr,3)/error;
/* Update LPC coefficients and total error */
lpc[i] = SHR32(r,3);
for (j = 0; j < (i+1)>>1; j++)
{
opus_val32 tmp1, tmp2;
tmp1 = lpc[j];
tmp2 = lpc[i-1-j];
lpc[j] = tmp1 + MULT32_32_Q31(r,tmp2);
lpc[i-1-j] = tmp2 + MULT32_32_Q31(r,tmp1);
}
error = error - MULT32_32_Q31(MULT32_32_Q31(r,r),error);
/* Bail out once we get 30 dB gain */
#ifdef FIXED_POINT
if (error<SHR32(ac[0],10))
break;
#else
if (error<.001f*ac[0])
break;
#endif
}
}
#ifdef FIXED_POINT
for (i=0;i<p;i++)
_lpc[i] = ROUND16(lpc[i],16);
#endif
}
int rnn_autocorr(
const opus_val16 *x, /* in: [0...n-1] samples x */
opus_val32 *ac, /* out: [0...lag-1] ac values */
const opus_val16 *window,
int overlap,
int lag,
int n)
{
opus_val32 d;
int i, k;
int fastN=n-lag;
int shift;
const opus_val16 *xptr;
opus_val16 xx[PITCH_BUF_SIZE/2];
celt_assert(n>0);
celt_assert(n<=PITCH_BUF_SIZE/2)
celt_assert(overlap>=0);
if (overlap == 0)
{
xptr = x;
} else {
for (i=0;i<n;i++)
xx[i] = x[i];
for (i=0;i<overlap;i++)
{
xx[i] = MULT16_16_Q15(x[i],window[i]);
xx[n-i-1] = MULT16_16_Q15(x[n-i-1],window[i]);
}
xptr = xx;
}
shift=0;
#ifdef FIXED_POINT
{
opus_val32 ac0;
ac0 = 1+(n<<7);
if (n&1) ac0 += SHR32(MULT16_16(xptr[0],xptr[0]),9);
for(i=(n&1);i<n;i+=2)
{
ac0 += SHR32(MULT16_16(xptr[i],xptr[i]),9);
ac0 += SHR32(MULT16_16(xptr[i+1],xptr[i+1]),9);
}
shift = celt_ilog2(ac0)-30+10;
shift = (shift)/2;
if (shift>0)
{
for(i=0;i<n;i++)
xx[i] = PSHR32(xptr[i], shift);
xptr = xx;
} else
shift = 0;
}
#endif
rnn_pitch_xcorr(xptr, xptr, ac, fastN, lag+1);
for (k=0;k<=lag;k++)
{
for (i = k+fastN, d = 0; i < n; i++)
d = MAC16_16(d, xptr[i], xptr[i-k]);
ac[k] += d;
}
#ifdef FIXED_POINT
shift = 2*shift;
if (shift<=0)
ac[0] += SHL32((opus_int32)1, -shift);
if (ac[0] < 268435456)
{
int shift2 = 29 - EC_ILOG(ac[0]);
for (i=0;i<=lag;i++)
ac[i] = SHL32(ac[i], shift2);
shift -= shift2;
} else if (ac[0] >= 536870912)
{
int shift2=1;
if (ac[0] >= 1073741824)
shift2++;
for (i=0;i<=lag;i++)
ac[i] = SHR32(ac[i], shift2);
shift += shift2;
}
#endif
return shift;
}

45
third_party/rnnoise/src/celt_lpc.h vendored Normal file
View File

@@ -0,0 +1,45 @@
/* Copyright (c) 2009-2010 Xiph.Org Foundation
Written by Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef PLC_H
#define PLC_H
#include "arch.h"
#include "common.h"
#if defined(OPUS_X86_MAY_HAVE_SSE4_1)
#include "x86/celt_lpc_sse.h"
#endif
#define LPC_ORDER 24
void rnn_lpc(opus_val16 *_lpc, const opus_val32 *ac, int p);
int rnn_autocorr(const opus_val16 *x, opus_val32 *ac,
const opus_val16 *window, int overlap, int lag, int n);
#endif /* PLC_H */

56
third_party/rnnoise/src/common.h vendored Normal file
View File

@@ -0,0 +1,56 @@
#ifndef COMMON_H
#define COMMON_H
#include "stdlib.h"
#include "string.h"
#define RNN_INLINE inline
#define OPUS_INLINE inline
/** RNNoise wrapper for malloc(). To do your own dynamic allocation, all you need t
o do is replace this function and rnnoise_free */
#ifndef OVERRIDE_RNNOISE_ALLOC
static RNN_INLINE void *rnnoise_alloc (size_t size)
{
return malloc(size);
}
#endif
/** RNNoise wrapper for free(). To do your own dynamic allocation, all you need to do is replace this function and rnnoise_alloc */
#ifndef OVERRIDE_RNNOISE_FREE
static RNN_INLINE void rnnoise_free (void *ptr)
{
free(ptr);
}
#endif
/** Copy n elements from src to dst. The 0* term provides compile-time type checking */
#ifndef OVERRIDE_RNN_COPY
#define RNN_COPY(dst, src, n) (memcpy((dst), (src), (n)*sizeof(*(dst)) + 0*((dst)-(src)) ))
#endif
/** Copy n elements from src to dst, allowing overlapping regions. The 0* term
provides compile-time type checking */
#ifndef OVERRIDE_RNN_MOVE
#define RNN_MOVE(dst, src, n) (memmove((dst), (src), (n)*sizeof(*(dst)) + 0*((dst)-(src)) ))
#endif
/** Set n elements of dst to zero */
#ifndef OVERRIDE_RNN_CLEAR
#define RNN_CLEAR(dst, n) (memset((dst), 0, (n)*sizeof(*(dst))))
#endif
# if !defined(OPUS_GNUC_PREREQ)
# if defined(__GNUC__)&&defined(__GNUC_MINOR__)
# define OPUS_GNUC_PREREQ(_maj,_min) \
((__GNUC__<<16)+__GNUC_MINOR__>=((_maj)<<16)+(_min))
# else
# define OPUS_GNUC_PREREQ(_maj,_min) 0
# endif
# endif
#endif

53
third_party/rnnoise/src/cpu_support.h vendored Normal file
View File

@@ -0,0 +1,53 @@
/* Copyright (c) 2010 Xiph.Org Foundation
* Copyright (c) 2013 Parrot */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef CPU_SUPPORT_H
#define CPU_SUPPORT_H
#include "opus_types.h"
#include "common.h"
#ifdef RNN_ENABLE_X86_RTCD
#include "x86/x86cpu.h"
/* We currently support 5 x86 variants:
* arch[0] -> sse2
* arch[1] -> sse4.1
* arch[2] -> avx2
*/
#define OPUS_ARCHMASK 3
int rnn_select_arch(void);
#else
#define OPUS_ARCHMASK 0
static OPUS_INLINE int rnn_select_arch(void)
{
return 0;
}
#endif
#endif

505
third_party/rnnoise/src/denoise.c vendored Normal file
View File

@@ -0,0 +1,505 @@
/* Copyright (c) 2024 Jean-Marc Valin
* Copyright (c) 2018 Gregor Richards
* Copyright (c) 2017 Mozilla */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "kiss_fft.h"
#include "common.h"
#include "denoise.h"
#include <math.h>
#include "rnnoise.h"
#include "pitch.h"
#include "arch.h"
#include "rnn.h"
#include "cpu_support.h"
#define SQUARE(x) ((x)*(x))
#ifndef TRAINING
#define TRAINING 0
#endif
/* ERB bandwidths going in reverse from 20 kHz and then replacing the 700 and 800
with just 750 because having 32 bands is convenient for the DNN.
B(1)=400;
for k=2:35
B(k) = B(k-1) - max(2, round(24.7*(4.37*B(k-1)/20+1)/50));
end
printf("%d, ", B(end:-1:1));
printf("\n")
*/
const int eband20ms[NB_BANDS+2] = {
/*0 100 200 300 400 500 600 750 900 1.1 1.2 1.4 1.6 1.8 2.1 2.4 2.7 3.0 3.4 3.9 4.4 4.9 5.5 6.2 7.0 7.9 8.8 9.9 11.2 12.6 14.1 15.9 17.8 20.0*/
0, 2, 4, 6, 8, 10, 12, 15, 18, 21, 24, 28, 32, 36, 41, 47, 53, 60, 68, 77, 87, 98, 110, 124, 140, 157, 176, 198, 223, 251, 282, 317, 356, 400};
struct DenoiseState {
RNNoise model;
#if !TRAINING
int arch;
#endif
float analysis_mem[FRAME_SIZE];
int memid;
float synthesis_mem[FRAME_SIZE];
float pitch_buf[PITCH_BUF_SIZE];
float pitch_enh_buf[PITCH_BUF_SIZE];
float last_gain;
int last_period;
float mem_hp_x[2];
float lastg[NB_BANDS];
RNNState rnn;
kiss_fft_cpx delayed_X[FREQ_SIZE];
kiss_fft_cpx delayed_P[FREQ_SIZE];
float delayed_Ex[NB_BANDS], delayed_Ep[NB_BANDS];
float delayed_Exp[NB_BANDS];
};
static void compute_band_energy(float *bandE, const kiss_fft_cpx *X) {
int i;
float sum[NB_BANDS+2] = {0};
for (i=0;i<NB_BANDS+1;i++)
{
int j;
int band_size;
band_size = eband20ms[i+1]-eband20ms[i];
for (j=0;j<band_size;j++) {
float tmp;
float frac = (float)j/band_size;
tmp = SQUARE(X[eband20ms[i] + j].r);
tmp += SQUARE(X[eband20ms[i] + j].i);
sum[i] += (1-frac)*tmp;
sum[i+1] += frac*tmp;
}
}
sum[1] = (sum[0]+sum[1])*2/3;
sum[NB_BANDS] = (sum[NB_BANDS]+sum[NB_BANDS+1])*2/3;
for (i=0;i<NB_BANDS;i++)
{
bandE[i] = sum[i+1];
}
}
static void compute_band_corr(float *bandE, const kiss_fft_cpx *X, const kiss_fft_cpx *P) {
int i;
float sum[NB_BANDS+2] = {0};
for (i=0;i<NB_BANDS+1;i++)
{
int j;
int band_size;
band_size = eband20ms[i+1]-eband20ms[i];
for (j=0;j<band_size;j++) {
float tmp;
float frac = (float)j/band_size;
tmp = X[eband20ms[i] + j].r * P[eband20ms[i] + j].r;
tmp += X[eband20ms[i] + j].i * P[eband20ms[i] + j].i;
sum[i] += (1-frac)*tmp;
sum[i+1] += frac*tmp;
}
}
sum[1] = (sum[0]+sum[1])*2/3;
sum[NB_BANDS] = (sum[NB_BANDS]+sum[NB_BANDS+1])*2/3;
for (i=0;i<NB_BANDS;i++)
{
bandE[i] = sum[i+1];
}
}
static void interp_band_gain(float *g, const float *bandE) {
int i,j;
memset(g, 0, FREQ_SIZE);
for (i=1;i<NB_BANDS;i++)
{
int band_size;
band_size = eband20ms[i+1]-eband20ms[i];
for (j=0;j<band_size;j++) {
float frac = (float)j/band_size;
g[eband20ms[i] + j] = (1-frac)*bandE[i-1] + frac*bandE[i];
}
}
for (j=0;j<eband20ms[1];j++) g[j] = bandE[0];
for (j=eband20ms[NB_BANDS];j<eband20ms[NB_BANDS+1];j++) g[j] = bandE[NB_BANDS-1];
}
extern const float rnn_dct_table[];
extern const kiss_fft_state rnn_kfft;
extern const float rnn_half_window[];
static void dct(float *out, const float *in) {
int i;
for (i=0;i<NB_BANDS;i++) {
int j;
float sum = 0;
for (j=0;j<NB_BANDS;j++) {
sum += in[j] * rnn_dct_table[j*NB_BANDS + i];
}
out[i] = sum*sqrt(2./22);
}
}
#if 0
static void idct(float *out, const float *in) {
int i;
for (i=0;i<NB_BANDS;i++) {
int j;
float sum = 0;
for (j=0;j<NB_BANDS;j++) {
sum += in[j] * rnn_dct_table[i*NB_BANDS + j];
}
out[i] = sum*sqrt(2./22);
}
}
#endif
static void forward_transform(kiss_fft_cpx *out, const float *in) {
int i;
kiss_fft_cpx x[WINDOW_SIZE];
kiss_fft_cpx y[WINDOW_SIZE];
for (i=0;i<WINDOW_SIZE;i++) {
x[i].r = in[i];
x[i].i = 0;
}
rnn_fft(&rnn_kfft, x, y, 0);
for (i=0;i<FREQ_SIZE;i++) {
out[i] = y[i];
}
}
static void inverse_transform(float *out, const kiss_fft_cpx *in) {
int i;
kiss_fft_cpx x[WINDOW_SIZE];
kiss_fft_cpx y[WINDOW_SIZE];
for (i=0;i<FREQ_SIZE;i++) {
x[i] = in[i];
}
for (;i<WINDOW_SIZE;i++) {
x[i].r = x[WINDOW_SIZE - i].r;
x[i].i = -x[WINDOW_SIZE - i].i;
}
rnn_fft(&rnn_kfft, x, y, 0);
/* output in reverse order for IFFT. */
out[0] = WINDOW_SIZE*y[0].r;
for (i=1;i<WINDOW_SIZE;i++) {
out[i] = WINDOW_SIZE*y[WINDOW_SIZE - i].r;
}
}
static void apply_window(float *x) {
int i;
for (i=0;i<FRAME_SIZE;i++) {
x[i] *= rnn_half_window[i];
x[WINDOW_SIZE - 1 - i] *= rnn_half_window[i];
}
}
struct RNNModel {
/* Set either blob or const_blob. */
const void *const_blob;
void *blob;
int blob_len;
FILE *file;
};
RNNModel *rnnoise_model_from_buffer(const void *ptr, int len) {
RNNModel *model;
model = malloc(sizeof(*model));
model->blob = NULL;
model->const_blob = ptr;
model->blob_len = len;
return model;
}
RNNModel *rnnoise_model_from_filename(const char *filename) {
RNNModel *model;
FILE *f = fopen(filename, "rb");
model = rnnoise_model_from_file(f);
model->file = f;
return model;
}
RNNModel *rnnoise_model_from_file(FILE *f) {
RNNModel *model;
model = malloc(sizeof(*model));
model->file = NULL;
fseek(f, 0, SEEK_END);
model->blob_len = ftell(f);
fseek(f, 0, SEEK_SET);
model->const_blob = NULL;
model->blob = malloc(model->blob_len);
if (fread(model->blob, model->blob_len, 1, f) != 1)
{
rnnoise_model_free(model);
return NULL;
}
return model;
}
void rnnoise_model_free(RNNModel *model) {
if (model->file != NULL) fclose(model->file);
if (model->blob != NULL) free(model->blob);
free(model);
}
int rnnoise_get_size(void) {
return sizeof(DenoiseState);
}
int rnnoise_get_frame_size(void) {
return FRAME_SIZE;
}
int rnnoise_init(DenoiseState *st, RNNModel *model) {
memset(st, 0, sizeof(*st));
#if !TRAINING
if (model != NULL) {
WeightArray *list;
int ret = 1;
parse_weights(&list, model->blob ? model->blob : model->const_blob, model->blob_len);
if (list != NULL) {
ret = init_rnnoise(&st->model, list);
opus_free(list);
}
if (ret != 0) return -1;
}
#ifndef USE_WEIGHTS_FILE
else {
int ret = init_rnnoise(&st->model, rnnoise_arrays);
if (ret != 0) return -1;
}
#endif
st->arch = rnn_select_arch();
#else
(void)model;
#endif
return 0;
}
DenoiseState *rnnoise_create(RNNModel *model) {
int ret;
DenoiseState *st;
st = malloc(rnnoise_get_size());
ret = rnnoise_init(st, model);
if (ret != 0) {
free(st);
return NULL;
}
return st;
}
void rnnoise_destroy(DenoiseState *st) {
free(st);
}
#if TRAINING
extern int lowpass;
extern int band_lp;
#endif
void rnn_frame_analysis(DenoiseState *st, kiss_fft_cpx *X, float *Ex, const float *in) {
int i;
float x[WINDOW_SIZE];
RNN_COPY(x, st->analysis_mem, FRAME_SIZE);
for (i=0;i<FRAME_SIZE;i++) x[FRAME_SIZE + i] = in[i];
RNN_COPY(st->analysis_mem, in, FRAME_SIZE);
apply_window(x);
forward_transform(X, x);
#if TRAINING
for (i=lowpass;i<FREQ_SIZE;i++)
X[i].r = X[i].i = 0;
#endif
compute_band_energy(Ex, X);
}
int rnn_compute_frame_features(DenoiseState *st, kiss_fft_cpx *X, kiss_fft_cpx *P,
float *Ex, float *Ep, float *Exp, float *features, const float *in) {
int i;
float E = 0;
float Ly[NB_BANDS];
float p[WINDOW_SIZE];
float pitch_buf[PITCH_BUF_SIZE>>1];
int pitch_index;
float gain;
float *(pre[1]);
float follow, logMax;
rnn_frame_analysis(st, X, Ex, in);
RNN_MOVE(st->pitch_buf, &st->pitch_buf[FRAME_SIZE], PITCH_BUF_SIZE-FRAME_SIZE);
RNN_COPY(&st->pitch_buf[PITCH_BUF_SIZE-FRAME_SIZE], in, FRAME_SIZE);
pre[0] = &st->pitch_buf[0];
rnn_pitch_downsample(pre, pitch_buf, PITCH_BUF_SIZE, 1);
rnn_pitch_search(pitch_buf+(PITCH_MAX_PERIOD>>1), pitch_buf, PITCH_FRAME_SIZE,
PITCH_MAX_PERIOD-3*PITCH_MIN_PERIOD, &pitch_index);
pitch_index = PITCH_MAX_PERIOD-pitch_index;
gain = rnn_remove_doubling(pitch_buf, PITCH_MAX_PERIOD, PITCH_MIN_PERIOD,
PITCH_FRAME_SIZE, &pitch_index, st->last_period, st->last_gain);
st->last_period = pitch_index;
st->last_gain = gain;
for (i=0;i<WINDOW_SIZE;i++)
p[i] = st->pitch_buf[PITCH_BUF_SIZE-WINDOW_SIZE-pitch_index+i];
apply_window(p);
forward_transform(P, p);
compute_band_energy(Ep, P);
compute_band_corr(Exp, X, P);
for (i=0;i<NB_BANDS;i++) Exp[i] = Exp[i]/sqrt(.001+Ex[i]*Ep[i]);
dct(&features[NB_BANDS], Exp);
features[2*NB_BANDS] = .01*(pitch_index-300);
logMax = -2;
follow = -2;
for (i=0;i<NB_BANDS;i++) {
Ly[i] = log10(1e-2+Ex[i]);
Ly[i] = MAX16(logMax-7, MAX16(follow-1.5, Ly[i]));
logMax = MAX16(logMax, Ly[i]);
follow = MAX16(follow-1.5, Ly[i]);
E += Ex[i];
}
if (!TRAINING && E < 0.04) {
/* If there's no audio, avoid messing up the state. */
RNN_CLEAR(features, NB_FEATURES);
return 1;
}
dct(features, Ly);
features[0] -= 12;
features[1] -= 4;
return TRAINING && E < 0.1;
}
static void frame_synthesis(DenoiseState *st, float *out, const kiss_fft_cpx *y) {
float x[WINDOW_SIZE];
int i;
inverse_transform(x, y);
apply_window(x);
for (i=0;i<FRAME_SIZE;i++) out[i] = x[i] + st->synthesis_mem[i];
RNN_COPY(st->synthesis_mem, &x[FRAME_SIZE], FRAME_SIZE);
}
void rnn_biquad(float *y, float mem[2], const float *x, const float *b, const float *a, int N) {
int i;
for (i=0;i<N;i++) {
float xi, yi;
xi = x[i];
yi = x[i] + mem[0];
mem[0] = mem[1] + (b[0]*(double)xi - a[0]*(double)yi);
mem[1] = (b[1]*(double)xi - a[1]*(double)yi);
y[i] = yi;
}
}
void rnn_pitch_filter(kiss_fft_cpx *X, const kiss_fft_cpx *P, const float *Ex, const float *Ep,
const float *Exp, const float *g) {
int i;
float r[NB_BANDS];
float rf[FREQ_SIZE] = {0};
float newE[NB_BANDS];
float norm[NB_BANDS];
float normf[FREQ_SIZE]={0};
for (i=0;i<NB_BANDS;i++) {
#if 0
if (Exp[i]>g[i]) r[i] = 1;
else r[i] = Exp[i]*(1-g[i])/(.001 + g[i]*(1-Exp[i]));
r[i] = MIN16(1, MAX16(0, r[i]));
#else
if (Exp[i]>g[i]) r[i] = 1;
else r[i] = SQUARE(Exp[i])*(1-SQUARE(g[i]))/(.001 + SQUARE(g[i])*(1-SQUARE(Exp[i])));
r[i] = sqrt(MIN16(1, MAX16(0, r[i])));
#endif
r[i] *= sqrt(Ex[i]/(1e-8+Ep[i]));
}
interp_band_gain(rf, r);
for (i=0;i<FREQ_SIZE;i++) {
X[i].r += rf[i]*P[i].r;
X[i].i += rf[i]*P[i].i;
}
compute_band_energy(newE, X);
for (i=0;i<NB_BANDS;i++) {
norm[i] = sqrt(Ex[i]/(1e-8+newE[i]));
}
interp_band_gain(normf, norm);
for (i=0;i<FREQ_SIZE;i++) {
X[i].r *= normf[i];
X[i].i *= normf[i];
}
}
float rnnoise_process_frame(DenoiseState *st, float *out, const float *in) {
int i;
kiss_fft_cpx X[FREQ_SIZE];
kiss_fft_cpx P[FREQ_SIZE];
float x[FRAME_SIZE];
float Ex[NB_BANDS], Ep[NB_BANDS];
float Exp[NB_BANDS];
float features[NB_FEATURES];
float g[NB_BANDS];
float gf[FREQ_SIZE]={1};
float vad_prob = 0;
int silence;
static const float a_hp[2] = {-1.99599, 0.99600};
static const float b_hp[2] = {-2, 1};
rnn_biquad(x, st->mem_hp_x, in, b_hp, a_hp, FRAME_SIZE);
silence = rnn_compute_frame_features(st, X, P, Ex, Ep, Exp, features, x);
if (!silence) {
#if !TRAINING
compute_rnn(&st->model, &st->rnn, g, &vad_prob, features, st->arch);
#endif
rnn_pitch_filter(st->delayed_X, st->delayed_P, st->delayed_Ex, st->delayed_Ep, st->delayed_Exp, g);
for (i=0;i<NB_BANDS;i++) {
float alpha = .6f;
/* Cap the decay at 0.6 per frame, corresponding to an RT60 of 135 ms.
That avoids unnaturally quick attenuation. */
g[i] = MAX16(g[i], alpha*st->lastg[i]);
/* Compensate for energy change across frame when computing the threshold gain.
Avoids leaking noise when energy increases (e.g. transient noise). */
st->lastg[i] = MIN16(1.f, g[i]*(st->delayed_Ex[i]+1e-3)/(Ex[i]+1e-3));
}
interp_band_gain(gf, g);
#if 1
for (i=0;i<FREQ_SIZE;i++) {
st->delayed_X[i].r *= gf[i];
st->delayed_X[i].i *= gf[i];
}
#endif
}
frame_synthesis(st, out, st->delayed_X);
RNN_COPY(st->delayed_X, X, FREQ_SIZE);
RNN_COPY(st->delayed_P, P, FREQ_SIZE);
RNN_COPY(st->delayed_Ex, Ex, NB_BANDS);
RNN_COPY(st->delayed_Ep, Ep, NB_BANDS);
RNN_COPY(st->delayed_Exp, Exp, NB_BANDS);
return vad_prob;
}

56
third_party/rnnoise/src/denoise.h vendored Normal file
View File

@@ -0,0 +1,56 @@
/* Copyright (c) 2017 Mozilla */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include "rnnoise.h"
#include "kiss_fft.h"
#include "nnet.h"
#define FRAME_SIZE 480
#define WINDOW_SIZE (2*FRAME_SIZE)
#define FREQ_SIZE (FRAME_SIZE + 1)
#define NB_BANDS 32
#define NB_FEATURES (2*NB_BANDS+1)
#define PITCH_MIN_PERIOD 60
#define PITCH_MAX_PERIOD 768
#define PITCH_FRAME_SIZE 960
#define PITCH_BUF_SIZE (PITCH_MAX_PERIOD+PITCH_FRAME_SIZE)
extern const WeightArray rnnoise_arrays[];
extern const int eband20ms[];
void rnn_biquad(float *y, float mem[2], const float *x, const float *b, const float *a, int N);
void rnn_pitch_filter(kiss_fft_cpx *X, const kiss_fft_cpx *P, const float *Ex, const float *Ep,
const float *Exp, const float *g);
void rnn_frame_analysis(DenoiseState *st, kiss_fft_cpx *X, float *Ex, const float *in);
int rnn_compute_frame_features(DenoiseState *st, kiss_fft_cpx *X, kiss_fft_cpx *P,
float *Ex, float *Ep, float *Exp, float *features, const float *in);

601
third_party/rnnoise/src/kiss_fft.c vendored Normal file
View File

@@ -0,0 +1,601 @@
/*Copyright (c) 2003-2004, Mark Borgerding
Lots of modifications by Jean-Marc Valin
Copyright (c) 2005-2007, Xiph.Org Foundation
Copyright (c) 2008, Xiph.Org Foundation, CSIRO
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.*/
/* This code is originally from Mark Borgerding's KISS-FFT but has been
heavily modified to better suit Opus */
#ifndef SKIP_CONFIG_H
# ifdef HAVE_CONFIG_H
# include "config.h"
# endif
#endif
#include "_kiss_fft_guts.h"
#define CUSTOM_MODES
/* The guts header contains all the multiplication and addition macros that are defined for
complex numbers. It also declares the kf_ internal functions.
*/
static void kf_bfly2(
kiss_fft_cpx * Fout,
int m,
int N
)
{
kiss_fft_cpx * Fout2;
int i;
(void)m;
#ifdef CUSTOM_MODES
if (m==1)
{
celt_assert(m==1);
for (i=0;i<N;i++)
{
kiss_fft_cpx t;
Fout2 = Fout + 1;
t = *Fout2;
C_SUB( *Fout2 , *Fout , t );
C_ADDTO( *Fout , t );
Fout += 2;
}
} else
#endif
{
opus_val16 tw;
tw = QCONST16(0.7071067812f, 15);
/* We know that m==4 here because the radix-2 is just after a radix-4 */
celt_assert(m==4);
for (i=0;i<N;i++)
{
kiss_fft_cpx t;
Fout2 = Fout + 4;
t = Fout2[0];
C_SUB( Fout2[0] , Fout[0] , t );
C_ADDTO( Fout[0] , t );
t.r = S_MUL(ADD32_ovflw(Fout2[1].r, Fout2[1].i), tw);
t.i = S_MUL(SUB32_ovflw(Fout2[1].i, Fout2[1].r), tw);
C_SUB( Fout2[1] , Fout[1] , t );
C_ADDTO( Fout[1] , t );
t.r = Fout2[2].i;
t.i = -Fout2[2].r;
C_SUB( Fout2[2] , Fout[2] , t );
C_ADDTO( Fout[2] , t );
t.r = S_MUL(SUB32_ovflw(Fout2[3].i, Fout2[3].r), tw);
t.i = S_MUL(NEG32_ovflw(ADD32_ovflw(Fout2[3].i, Fout2[3].r)), tw);
C_SUB( Fout2[3] , Fout[3] , t );
C_ADDTO( Fout[3] , t );
Fout += 8;
}
}
}
static void kf_bfly4(
kiss_fft_cpx * Fout,
const size_t fstride,
const kiss_fft_state *st,
int m,
int N,
int mm
)
{
int i;
if (m==1)
{
/* Degenerate case where all the twiddles are 1. */
for (i=0;i<N;i++)
{
kiss_fft_cpx scratch0, scratch1;
C_SUB( scratch0 , *Fout, Fout[2] );
C_ADDTO(*Fout, Fout[2]);
C_ADD( scratch1 , Fout[1] , Fout[3] );
C_SUB( Fout[2], *Fout, scratch1 );
C_ADDTO( *Fout , scratch1 );
C_SUB( scratch1 , Fout[1] , Fout[3] );
Fout[1].r = ADD32_ovflw(scratch0.r, scratch1.i);
Fout[1].i = SUB32_ovflw(scratch0.i, scratch1.r);
Fout[3].r = SUB32_ovflw(scratch0.r, scratch1.i);
Fout[3].i = ADD32_ovflw(scratch0.i, scratch1.r);
Fout+=4;
}
} else {
int j;
kiss_fft_cpx scratch[6];
const kiss_twiddle_cpx *tw1,*tw2,*tw3;
const int m2=2*m;
const int m3=3*m;
kiss_fft_cpx * Fout_beg = Fout;
for (i=0;i<N;i++)
{
Fout = Fout_beg + i*mm;
tw3 = tw2 = tw1 = st->twiddles;
/* m is guaranteed to be a multiple of 4. */
for (j=0;j<m;j++)
{
C_MUL(scratch[0],Fout[m] , *tw1 );
C_MUL(scratch[1],Fout[m2] , *tw2 );
C_MUL(scratch[2],Fout[m3] , *tw3 );
C_SUB( scratch[5] , *Fout, scratch[1] );
C_ADDTO(*Fout, scratch[1]);
C_ADD( scratch[3] , scratch[0] , scratch[2] );
C_SUB( scratch[4] , scratch[0] , scratch[2] );
C_SUB( Fout[m2], *Fout, scratch[3] );
tw1 += fstride;
tw2 += fstride*2;
tw3 += fstride*3;
C_ADDTO( *Fout , scratch[3] );
Fout[m].r = ADD32_ovflw(scratch[5].r, scratch[4].i);
Fout[m].i = SUB32_ovflw(scratch[5].i, scratch[4].r);
Fout[m3].r = SUB32_ovflw(scratch[5].r, scratch[4].i);
Fout[m3].i = ADD32_ovflw(scratch[5].i, scratch[4].r);
++Fout;
}
}
}
}
#ifndef RADIX_TWO_ONLY
static void kf_bfly3(
kiss_fft_cpx * Fout,
const size_t fstride,
const kiss_fft_state *st,
int m,
int N,
int mm
)
{
int i;
size_t k;
const size_t m2 = 2*m;
const kiss_twiddle_cpx *tw1,*tw2;
kiss_fft_cpx scratch[5];
kiss_twiddle_cpx epi3;
kiss_fft_cpx * Fout_beg = Fout;
#ifdef FIXED_POINT
/*epi3.r = -16384;*/ /* Unused */
epi3.i = -28378;
#else
epi3 = st->twiddles[fstride*m];
#endif
for (i=0;i<N;i++)
{
Fout = Fout_beg + i*mm;
tw1=tw2=st->twiddles;
/* For non-custom modes, m is guaranteed to be a multiple of 4. */
k=m;
do {
C_MUL(scratch[1],Fout[m] , *tw1);
C_MUL(scratch[2],Fout[m2] , *tw2);
C_ADD(scratch[3],scratch[1],scratch[2]);
C_SUB(scratch[0],scratch[1],scratch[2]);
tw1 += fstride;
tw2 += fstride*2;
Fout[m].r = SUB32_ovflw(Fout->r, HALF_OF(scratch[3].r));
Fout[m].i = SUB32_ovflw(Fout->i, HALF_OF(scratch[3].i));
C_MULBYSCALAR( scratch[0] , epi3.i );
C_ADDTO(*Fout,scratch[3]);
Fout[m2].r = ADD32_ovflw(Fout[m].r, scratch[0].i);
Fout[m2].i = SUB32_ovflw(Fout[m].i, scratch[0].r);
Fout[m].r = SUB32_ovflw(Fout[m].r, scratch[0].i);
Fout[m].i = ADD32_ovflw(Fout[m].i, scratch[0].r);
++Fout;
} while(--k);
}
}
#ifndef OVERRIDE_kf_bfly5
static void kf_bfly5(
kiss_fft_cpx * Fout,
const size_t fstride,
const kiss_fft_state *st,
int m,
int N,
int mm
)
{
kiss_fft_cpx *Fout0,*Fout1,*Fout2,*Fout3,*Fout4;
int i, u;
kiss_fft_cpx scratch[13];
const kiss_twiddle_cpx *tw;
kiss_twiddle_cpx ya,yb;
kiss_fft_cpx * Fout_beg = Fout;
#ifdef FIXED_POINT
ya.r = 10126;
ya.i = -31164;
yb.r = -26510;
yb.i = -19261;
#else
ya = st->twiddles[fstride*m];
yb = st->twiddles[fstride*2*m];
#endif
tw=st->twiddles;
for (i=0;i<N;i++)
{
Fout = Fout_beg + i*mm;
Fout0=Fout;
Fout1=Fout0+m;
Fout2=Fout0+2*m;
Fout3=Fout0+3*m;
Fout4=Fout0+4*m;
/* For non-custom modes, m is guaranteed to be a multiple of 4. */
for ( u=0; u<m; ++u ) {
scratch[0] = *Fout0;
C_MUL(scratch[1] ,*Fout1, tw[u*fstride]);
C_MUL(scratch[2] ,*Fout2, tw[2*u*fstride]);
C_MUL(scratch[3] ,*Fout3, tw[3*u*fstride]);
C_MUL(scratch[4] ,*Fout4, tw[4*u*fstride]);
C_ADD( scratch[7],scratch[1],scratch[4]);
C_SUB( scratch[10],scratch[1],scratch[4]);
C_ADD( scratch[8],scratch[2],scratch[3]);
C_SUB( scratch[9],scratch[2],scratch[3]);
Fout0->r = ADD32_ovflw(Fout0->r, ADD32_ovflw(scratch[7].r, scratch[8].r));
Fout0->i = ADD32_ovflw(Fout0->i, ADD32_ovflw(scratch[7].i, scratch[8].i));
scratch[5].r = ADD32_ovflw(scratch[0].r, ADD32_ovflw(S_MUL(scratch[7].r,ya.r), S_MUL(scratch[8].r,yb.r)));
scratch[5].i = ADD32_ovflw(scratch[0].i, ADD32_ovflw(S_MUL(scratch[7].i,ya.r), S_MUL(scratch[8].i,yb.r)));
scratch[6].r = ADD32_ovflw(S_MUL(scratch[10].i,ya.i), S_MUL(scratch[9].i,yb.i));
scratch[6].i = NEG32_ovflw(ADD32_ovflw(S_MUL(scratch[10].r,ya.i), S_MUL(scratch[9].r,yb.i)));
C_SUB(*Fout1,scratch[5],scratch[6]);
C_ADD(*Fout4,scratch[5],scratch[6]);
scratch[11].r = ADD32_ovflw(scratch[0].r, ADD32_ovflw(S_MUL(scratch[7].r,yb.r), S_MUL(scratch[8].r,ya.r)));
scratch[11].i = ADD32_ovflw(scratch[0].i, ADD32_ovflw(S_MUL(scratch[7].i,yb.r), S_MUL(scratch[8].i,ya.r)));
scratch[12].r = SUB32_ovflw(S_MUL(scratch[9].i,ya.i), S_MUL(scratch[10].i,yb.i));
scratch[12].i = SUB32_ovflw(S_MUL(scratch[10].r,yb.i), S_MUL(scratch[9].r,ya.i));
C_ADD(*Fout2,scratch[11],scratch[12]);
C_SUB(*Fout3,scratch[11],scratch[12]);
++Fout0;++Fout1;++Fout2;++Fout3;++Fout4;
}
}
}
#endif /* OVERRIDE_kf_bfly5 */
#endif
#ifdef CUSTOM_MODES
static
void compute_bitrev_table(
int Fout,
opus_int32 *f,
const size_t fstride,
int in_stride,
opus_int16 * factors,
const kiss_fft_state *st
)
{
const int p=*factors++; /* the radix */
const int m=*factors++; /* stage's fft length/p */
/*printf ("fft %d %d %d %d %d %d\n", p*m, m, p, s2, fstride*in_stride, N);*/
if (m==1)
{
int j;
for (j=0;j<p;j++)
{
*f = Fout+j;
f += fstride*in_stride;
}
} else {
int j;
for (j=0;j<p;j++)
{
compute_bitrev_table( Fout , f, fstride*p, in_stride, factors,st);
f += fstride*in_stride;
Fout += m;
}
}
}
/* facbuf is populated by p1,m1,p2,m2, ...
where
p[i] * m[i] = m[i-1]
m0 = n */
static
int kf_factor(int n,opus_int16 * facbuf)
{
int p=4;
int i;
int stages=0;
int nbak = n;
/*factor out powers of 4, powers of 2, then any remaining primes */
do {
while (n % p) {
switch (p) {
case 4: p = 2; break;
case 2: p = 3; break;
default: p += 2; break;
}
if (p>32000 || (opus_int32)p*(opus_int32)p > n)
p = n; /* no more factors, skip to end */
}
n /= p;
#ifdef RADIX_TWO_ONLY
if (p!=2 && p != 4)
#else
if (p>5)
#endif
{
return 0;
}
facbuf[2*stages] = p;
if (p==2 && stages > 1)
{
facbuf[2*stages] = 4;
facbuf[2] = 2;
}
stages++;
} while (n > 1);
n = nbak;
/* Reverse the order to get the radix 4 at the end, so we can use the
fast degenerate case. It turns out that reversing the order also
improves the noise behaviour. */
for (i=0;i<stages/2;i++)
{
int tmp;
tmp = facbuf[2*i];
facbuf[2*i] = facbuf[2*(stages-i-1)];
facbuf[2*(stages-i-1)] = tmp;
}
for (i=0;i<stages;i++)
{
n /= facbuf[2*i];
facbuf[2*i+1] = n;
}
return 1;
}
static void compute_twiddles(kiss_twiddle_cpx *twiddles, int nfft)
{
int i;
#ifdef FIXED_POINT
for (i=0;i<nfft;++i) {
opus_val32 phase = -i;
kf_cexp2(twiddles+i, DIV32(SHL32(phase,17),nfft));
}
#else
for (i=0;i<nfft;++i) {
const double pi=3.14159265358979323846264338327;
double phase = ( -2*pi /nfft ) * i;
kf_cexp(twiddles+i, phase );
}
#endif
}
int rnn_fft_alloc_arch_c(kiss_fft_state *st) {
(void)st;
return 0;
}
/*
*
* Allocates all necessary storage space for the fft and ifft.
* The return value is a contiguous block of memory. As such,
* It can be freed with free().
* */
kiss_fft_state *rnn_fft_alloc_twiddles(int nfft,void * mem,size_t * lenmem,
const kiss_fft_state *base, int arch)
{
kiss_fft_state *st=NULL;
size_t memneeded = sizeof(struct kiss_fft_state); /* twiddle factors*/
if ( lenmem==NULL ) {
st = ( kiss_fft_state*)KISS_FFT_MALLOC( memneeded );
}else{
if (mem != NULL && *lenmem >= memneeded)
st = (kiss_fft_state*)mem;
*lenmem = memneeded;
}
if (st) {
opus_int32 *bitrev;
kiss_twiddle_cpx *twiddles;
st->nfft=nfft;
#ifdef FIXED_POINT
st->scale_shift = celt_ilog2(st->nfft);
if (st->nfft == 1<<st->scale_shift)
st->scale = Q15ONE;
else
st->scale = (1073741824+st->nfft/2)/st->nfft>>(15-st->scale_shift);
#else
st->scale = 1.f/nfft;
#endif
if (base != NULL)
{
st->twiddles = base->twiddles;
st->shift = 0;
while (st->shift < 32 && nfft<<st->shift != base->nfft)
st->shift++;
if (st->shift>=32)
goto fail;
} else {
st->twiddles = twiddles = (kiss_twiddle_cpx*)KISS_FFT_MALLOC(sizeof(kiss_twiddle_cpx)*nfft);
compute_twiddles(twiddles, nfft);
st->shift = -1;
}
if (!kf_factor(nfft,st->factors))
{
goto fail;
}
/* bitrev */
st->bitrev = bitrev = (opus_int32*)KISS_FFT_MALLOC(sizeof(opus_int32)*nfft);
if (st->bitrev==NULL)
goto fail;
compute_bitrev_table(0, bitrev, 1,1, st->factors,st);
/* Initialize architecture specific fft parameters */
if (rnn_fft_alloc_arch(st, arch))
goto fail;
}
return st;
fail:
rnn_fft_free(st, arch);
return NULL;
}
kiss_fft_state *rnn_fft_alloc(int nfft,void * mem,size_t * lenmem, int arch)
{
return rnn_fft_alloc_twiddles(nfft, mem, lenmem, NULL, arch);
}
void rnn_fft_free_arch_c(kiss_fft_state *st) {
(void)st;
}
void rnn_fft_free(const kiss_fft_state *cfg, int arch)
{
if (cfg)
{
rnn_fft_free_arch((kiss_fft_state *)cfg, arch);
opus_free((opus_int32*)cfg->bitrev);
if (cfg->shift < 0)
opus_free((kiss_twiddle_cpx*)cfg->twiddles);
opus_free((kiss_fft_state*)cfg);
}
}
#endif /* CUSTOM_MODES */
void rnn_fft_impl(const kiss_fft_state *st,kiss_fft_cpx *fout)
{
int m2, m;
int p;
int L;
int fstride[MAXFACTORS];
int i;
int shift;
/* st->shift can be -1 */
shift = st->shift>0 ? st->shift : 0;
fstride[0] = 1;
L=0;
do {
p = st->factors[2*L];
m = st->factors[2*L+1];
fstride[L+1] = fstride[L]*p;
L++;
} while(m!=1);
m = st->factors[2*L-1];
for (i=L-1;i>=0;i--)
{
if (i!=0)
m2 = st->factors[2*i-1];
else
m2 = 1;
switch (st->factors[2*i])
{
case 2:
kf_bfly2(fout, m, fstride[i]);
break;
case 4:
kf_bfly4(fout,fstride[i]<<shift,st,m, fstride[i], m2);
break;
#ifndef RADIX_TWO_ONLY
case 3:
kf_bfly3(fout,fstride[i]<<shift,st,m, fstride[i], m2);
break;
case 5:
kf_bfly5(fout,fstride[i]<<shift,st,m, fstride[i], m2);
break;
#endif
}
m = m2;
}
}
void rnn_fft_c(const kiss_fft_state *st,const kiss_fft_cpx *fin,kiss_fft_cpx *fout)
{
int i;
opus_val16 scale;
#ifdef FIXED_POINT
/* Allows us to scale with MULT16_32_Q16(), which is faster than
MULT16_32_Q15() on ARM. */
int scale_shift = st->scale_shift-1;
#endif
scale = st->scale;
celt_assert2 (fin != fout, "In-place FFT not supported");
/* Bit-reverse the input */
for (i=0;i<st->nfft;i++)
{
kiss_fft_cpx x = fin[i];
fout[st->bitrev[i]].r = SHR32(MULT16_32_Q16(scale, x.r), scale_shift);
fout[st->bitrev[i]].i = SHR32(MULT16_32_Q16(scale, x.i), scale_shift);
}
rnn_fft_impl(st, fout);
}
void rnn_ifft_c(const kiss_fft_state *st,const kiss_fft_cpx *fin,kiss_fft_cpx *fout)
{
int i;
celt_assert2 (fin != fout, "In-place FFT not supported");
/* Bit-reverse the input */
for (i=0;i<st->nfft;i++)
fout[st->bitrev[i]] = fin[i];
for (i=0;i<st->nfft;i++)
fout[i].i = -fout[i].i;
rnn_fft_impl(st, fout);
for (i=0;i<st->nfft;i++)
fout[i].i = -fout[i].i;
}

203
third_party/rnnoise/src/kiss_fft.h vendored Normal file
View File

@@ -0,0 +1,203 @@
/*Copyright (c) 2003-2004, Mark Borgerding
Lots of modifications by Jean-Marc Valin
Copyright (c) 2005-2007, Xiph.Org Foundation
Copyright (c) 2008, Xiph.Org Foundation, CSIRO
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.*/
#ifndef KISS_FFT_H
#define KISS_FFT_H
#include <stdlib.h>
#include <math.h>
#include "arch.h"
#include <stdlib.h>
#define opus_alloc(x) malloc(x)
#define opus_free(x) free(x)
#ifdef __cplusplus
extern "C" {
#endif
#ifdef USE_SIMD
# include <xmmintrin.h>
# define kiss_fft_scalar __m128
#define KISS_FFT_MALLOC(nbytes) memalign(16,nbytes)
#else
#define KISS_FFT_MALLOC opus_alloc
#endif
#ifdef FIXED_POINT
#include "arch.h"
# define kiss_fft_scalar opus_int32
# define kiss_twiddle_scalar opus_int16
#else
# ifndef kiss_fft_scalar
/* default is float */
# define kiss_fft_scalar float
# define kiss_twiddle_scalar float
# define KF_SUFFIX _celt_single
# endif
#endif
typedef struct {
kiss_fft_scalar r;
kiss_fft_scalar i;
}kiss_fft_cpx;
typedef struct {
kiss_twiddle_scalar r;
kiss_twiddle_scalar i;
}kiss_twiddle_cpx;
#define MAXFACTORS 8
/* e.g. an fft of length 128 has 4 factors
as far as kissfft is concerned
4*4*4*2
*/
typedef struct arch_fft_state{
int is_supported;
void *priv;
} arch_fft_state;
typedef struct kiss_fft_state{
int nfft;
opus_val16 scale;
#ifdef FIXED_POINT
int scale_shift;
#endif
int shift;
opus_int16 factors[2*MAXFACTORS];
const opus_int32 *bitrev;
const kiss_twiddle_cpx *twiddles;
arch_fft_state *arch_fft;
} kiss_fft_state;
#if defined(HAVE_ARM_NE10)
#include "arm/fft_arm.h"
#endif
/*typedef struct kiss_fft_state* kiss_fft_cfg;*/
/**
* opus_fft_alloc
*
* Initialize a FFT (or IFFT) algorithm's cfg/state buffer.
*
* typical usage: kiss_fft_cfg mycfg=opus_fft_alloc(1024,0,NULL,NULL);
*
* The return value from fft_alloc is a cfg buffer used internally
* by the fft routine or NULL.
*
* If lenmem is NULL, then opus_fft_alloc will allocate a cfg buffer using malloc.
* The returned value should be free()d when done to avoid memory leaks.
*
* The state can be placed in a user supplied buffer 'mem':
* If lenmem is not NULL and mem is not NULL and *lenmem is large enough,
* then the function places the cfg in mem and the size used in *lenmem
* and returns mem.
*
* If lenmem is not NULL and ( mem is NULL or *lenmem is not large enough),
* then the function returns NULL and places the minimum cfg
* buffer size in *lenmem.
* */
kiss_fft_state *rnn_fft_alloc_twiddles(int nfft,void * mem,size_t * lenmem, const kiss_fft_state *base, int arch);
kiss_fft_state *rnn_fft_alloc(int nfft,void * mem,size_t * lenmem, int arch);
/**
* opus_fft(cfg,in_out_buf)
*
* Perform an FFT on a complex input buffer.
* for a forward FFT,
* fin should be f[0] , f[1] , ... ,f[nfft-1]
* fout will be F[0] , F[1] , ... ,F[nfft-1]
* Note that each element is complex and can be accessed like
f[k].r and f[k].i
* */
void rnn_fft_c(const kiss_fft_state *cfg,const kiss_fft_cpx *fin,kiss_fft_cpx *fout);
void rnn_ifft_c(const kiss_fft_state *cfg,const kiss_fft_cpx *fin,kiss_fft_cpx *fout);
void rnn_fft_impl(const kiss_fft_state *st,kiss_fft_cpx *fout);
void rnn_ifft_impl(const kiss_fft_state *st,kiss_fft_cpx *fout);
void rnn_fft_free(const kiss_fft_state *cfg, int arch);
void rnn_fft_free_arch_c(kiss_fft_state *st);
int rnn_fft_alloc_arch_c(kiss_fft_state *st);
#if !defined(OVERRIDE_OPUS_FFT)
/* Is run-time CPU detection enabled on this platform? */
#if defined(OPUS_HAVE_RTCD) && (defined(HAVE_ARM_NE10))
extern int (*const OPUS_FFT_ALLOC_ARCH_IMPL[OPUS_ARCHMASK+1])(
kiss_fft_state *st);
#define opus_fft_alloc_arch(_st, arch) \
((*OPUS_FFT_ALLOC_ARCH_IMPL[(arch)&OPUS_ARCHMASK])(_st))
extern void (*const OPUS_FFT_FREE_ARCH_IMPL[OPUS_ARCHMASK+1])(
kiss_fft_state *st);
#define opus_fft_free_arch(_st, arch) \
((*OPUS_FFT_FREE_ARCH_IMPL[(arch)&OPUS_ARCHMASK])(_st))
extern void (*const OPUS_FFT[OPUS_ARCHMASK+1])(const kiss_fft_state *cfg,
const kiss_fft_cpx *fin, kiss_fft_cpx *fout);
#define opus_fft(_cfg, _fin, _fout, arch) \
((*OPUS_FFT[(arch)&OPUS_ARCHMASK])(_cfg, _fin, _fout))
extern void (*const OPUS_IFFT[OPUS_ARCHMASK+1])(const kiss_fft_state *cfg,
const kiss_fft_cpx *fin, kiss_fft_cpx *fout);
#define opus_ifft(_cfg, _fin, _fout, arch) \
((*OPUS_IFFT[(arch)&OPUS_ARCHMASK])(_cfg, _fin, _fout))
#else /* else for if defined(OPUS_HAVE_RTCD) && (defined(HAVE_ARM_NE10)) */
#define rnn_fft_alloc_arch(_st, arch) \
((void)(arch), rnn_fft_alloc_arch_c(_st))
#define rnn_fft_free_arch(_st, arch) \
((void)(arch), rnn_fft_free_arch_c(_st))
#define rnn_fft(_cfg, _fin, _fout, arch) \
((void)(arch), rnn_fft_c(_cfg, _fin, _fout))
#define rnn_ifft(_cfg, _fin, _fout, arch) \
((void)(arch), rnn_ifft_c(_cfg, _fin, _fout))
#endif /* end if defined(OPUS_HAVE_RTCD) && (defined(HAVE_ARM_NE10)) */
#endif /* end if !defined(OVERRIDE_OPUS_FFT) */
#ifdef __cplusplus
}
#endif
#endif

123
third_party/rnnoise/src/nnet.c vendored Normal file
View File

@@ -0,0 +1,123 @@
/* Copyright (c) 2018 Mozilla
2008-2011 Octasic Inc.
2012-2017 Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include <stdlib.h>
#include <math.h>
#include "opus_types.h"
#include "arch.h"
#include "nnet.h"
#include "common.h"
#include "vec.h"
#ifdef ENABLE_OSCE
#include "osce.h"
#endif
#ifdef NO_OPTIMIZATIONS
#if defined(_MSC_VER)
#pragma message ("Compiling without any vectorization. This code will be very slow")
#else
#warning Compiling without any vectorization. This code will be very slow
#endif
#endif
#define SOFTMAX_HACK
void compute_generic_dense(const LinearLayer *layer, float *output, const float *input, int activation, int arch)
{
compute_linear(layer, output, input, arch);
compute_activation(output, output, layer->nb_outputs, activation, arch);
}
#define MAX_RNN_NEURONS_ALL 1024
void compute_generic_gru(const LinearLayer *input_weights, const LinearLayer *recurrent_weights, float *state, const float *in, int arch)
{
int i;
int N;
float zrh[3*MAX_RNN_NEURONS_ALL];
float recur[3*MAX_RNN_NEURONS_ALL];
float *z;
float *r;
float *h;
celt_assert(3*recurrent_weights->nb_inputs == recurrent_weights->nb_outputs);
celt_assert(input_weights->nb_outputs == recurrent_weights->nb_outputs);
N = recurrent_weights->nb_inputs;
z = zrh;
r = &zrh[N];
h = &zrh[2*N];
celt_assert(recurrent_weights->nb_outputs <= 3*MAX_RNN_NEURONS_ALL);
celt_assert(in != state);
compute_linear(input_weights, zrh, in, arch);
compute_linear(recurrent_weights, recur, state, arch);
for (i=0;i<2*N;i++)
zrh[i] += recur[i];
compute_activation(zrh, zrh, 2*N, ACTIVATION_SIGMOID, arch);
for (i=0;i<N;i++)
h[i] += recur[2*N+i]*r[i];
compute_activation(h, h, N, ACTIVATION_TANH, arch);
for (i=0;i<N;i++)
h[i] = z[i]*state[i] + (1-z[i])*h[i];
for (i=0;i<N;i++)
state[i] = h[i];
}
void compute_glu(const LinearLayer *layer, float *output, const float *input, int arch)
{
int i;
float act2[MAX_INPUTS];
celt_assert(layer->nb_inputs == layer->nb_outputs);
compute_linear(layer, act2, input, arch);
compute_activation(act2, act2, layer->nb_outputs, ACTIVATION_SIGMOID, arch);
if (input == output) {
/* Give a vectorization hint to the compiler for the in-place case. */
for (i=0;i<layer->nb_outputs;i++) output[i] = output[i]*act2[i];
} else {
for (i=0;i<layer->nb_outputs;i++) output[i] = input[i]*act2[i];
}
}
#define MAX_CONV_INPUTS_ALL 1024
void compute_generic_conv1d(const LinearLayer *layer, float *output, float *mem, const float *input, int input_size, int activation, int arch)
{
float tmp[MAX_CONV_INPUTS_ALL];
celt_assert(input != output);
celt_assert(layer->nb_inputs <= MAX_CONV_INPUTS_ALL);
if (layer->nb_inputs!=input_size) RNN_COPY(tmp, mem, layer->nb_inputs-input_size);
RNN_COPY(&tmp[layer->nb_inputs-input_size], input, input_size);
compute_linear(layer, output, tmp, arch);
compute_activation(output, output, layer->nb_outputs, activation, arch);
if (layer->nb_inputs!=input_size) RNN_COPY(mem, &tmp[input_size], layer->nb_inputs-input_size);
}

169
third_party/rnnoise/src/nnet.h vendored Normal file
View File

@@ -0,0 +1,169 @@
/* Copyright (c) 2018 Mozilla
Copyright (c) 2017 Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef NNET_H_
#define NNET_H_
#include <stddef.h>
#include "opus_types.h"
#define ACTIVATION_LINEAR 0
#define ACTIVATION_SIGMOID 1
#define ACTIVATION_TANH 2
#define ACTIVATION_RELU 3
#define ACTIVATION_SOFTMAX 4
#define ACTIVATION_SWISH 5
#define WEIGHT_BLOB_VERSION 0
#define WEIGHT_BLOCK_SIZE 64
typedef struct {
const char *name;
int type;
int size;
const void *data;
} WeightArray;
#define WEIGHT_TYPE_float 0
#define WEIGHT_TYPE_int 1
#define WEIGHT_TYPE_qweight 2
#define WEIGHT_TYPE_int8 3
typedef struct {
char head[4];
int version;
int type;
int size;
int block_size;
char name[44];
} WeightHead;
/* Generic sparse affine transformation. */
typedef struct {
const float *bias;
const float *subias;
const opus_int8 *weights;
const float *float_weights;
const int *weights_idx;
const float *diag;
const float *scale;
int nb_inputs;
int nb_outputs;
} LinearLayer;
/* Generic sparse affine transformation. */
typedef struct {
const float *bias;
const float *float_weights;
int in_channels;
int out_channels;
int ktime;
int kheight;
} Conv2dLayer;
/* Changes some symbol names to add the rnn_ prefix so we don't get conflicts with Opus. */
#define linear_init rnn_linear_init
#define conv2d_init rnn_conv2d_init
#define compute_generic_dense rnn_compute_generic_dense
#define compute_generic_gru rnn_compute_generic_gru
#define compute_generic_conv1d rnn_compute_generic_conv1d
#define compute_glu rnn_compute_glu
#define parse_weights rnn_parse_weights
#define compute_linear_c rnn_compute_linear_c
#define compute_activation_c rnn_compute_activation_c
#define compute_conv2d_c rnn_compute_conv2d_c
#define compute_linear_sse4_1 rnn_compute_linear_sse4_1
#define compute_activation_sse4_1 rnn_compute_activation_sse4_1
#define compute_conv2d_sse4_1 rnn_compute_conv2d_sse4_1
#define compute_linear_avx2 rnn_compute_linear_avx2
#define compute_activation_avx2 rnn_compute_activation_avx2
#define compute_conv2d_avx2 rnn_compute_conv2d_avx2
void compute_generic_dense(const LinearLayer *layer, float *output, const float *input, int activation, int arch);
void compute_generic_gru(const LinearLayer *input_weights, const LinearLayer *recurrent_weights, float *state, const float *in, int arch);
void compute_generic_conv1d(const LinearLayer *layer, float *output, float *mem, const float *input, int input_size, int activation, int arch);
void compute_glu(const LinearLayer *layer, float *output, const float *input, int arch);
int parse_weights(WeightArray **list, const void *data, int len);
int linear_init(LinearLayer *layer, const WeightArray *arrays,
const char *bias,
const char *subias,
const char *weights,
const char *float_weights,
const char *weights_idx,
const char *diag,
const char *scale,
int nb_inputs,
int nb_outputs);
int conv2d_init(Conv2dLayer *layer, const WeightArray *arrays,
const char *bias,
const char *float_weights,
int in_channels,
int out_channels,
int ktime,
int kheight);
void compute_linear_c(const LinearLayer *linear, float *out, const float *in);
void compute_activation_c(float *output, const float *input, int N, int activation);
void compute_conv2d_c(const Conv2dLayer *conv, float *out, float *mem, const float *in, int height, int hstride, int activation);
#ifdef RNN_ENABLE_X86_RTCD
#include "x86/dnn_x86.h"
#endif
#ifndef OVERRIDE_COMPUTE_LINEAR
#define compute_linear(linear, out, in, arch) ((void)(arch),compute_linear_c(linear, out, in))
#endif
#ifndef OVERRIDE_COMPUTE_ACTIVATION
#define compute_activation(output, input, N, activation, arch) ((void)(arch),compute_activation_c(output, input, N, activation))
#endif
#ifndef OVERRIDE_COMPUTE_CONV2D
#define compute_conv2d(conv, out, mem, in, height, hstride, activation, arch) ((void)(arch),compute_conv2d_c(conv, out, mem, in, height, hstride, activation))
#endif
#if defined(__x86_64__) && !defined(RNN_ENABLE_X86_RTCD) && !defined(__AVX2__)
#if defined(_MSC_VER)
#pragma message ("Only SSE and SSE2 are available. On newer machines, enable SSSE3/AVX/AVX2 to get better performance")
#else
#warning "Only SSE and SSE2 are available. On newer machines, enable SSSE3/AVX/AVX2 using -march= to get better performance"
#endif
#endif
#endif /* NNET_H_ */

257
third_party/rnnoise/src/nnet_arch.h vendored Normal file
View File

@@ -0,0 +1,257 @@
/* Copyright (c) 2018-2019 Mozilla
2023 Amazon */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef NNET_ARCH_H
#define NNET_ARCH_H
#include "nnet.h"
#include "arch.h"
#include "common.h"
#include "vec.h"
#define CAT_SUFFIX2(a,b) a ## b
#define CAT_SUFFIX(a,b) CAT_SUFFIX2(a, b)
#define RTCD_SUF(name) CAT_SUFFIX(name, RTCD_ARCH)
# if !defined(OPUS_GNUC_PREREQ)
# if defined(__GNUC__)&&defined(__GNUC_MINOR__)
# define OPUS_GNUC_PREREQ(_maj,_min) \
((__GNUC__<<16)+__GNUC_MINOR__>=((_maj)<<16)+(_min))
# else
# define OPUS_GNUC_PREREQ(_maj,_min) 0
# endif
# endif
/* Force vectorization on for DNN code because some of the loops rely on
compiler vectorization rather than explicitly using intrinsics. */
#if OPUS_GNUC_PREREQ(5,1)
#define GCC_POP_OPTIONS
#pragma GCC push_options
#pragma GCC optimize("tree-vectorize")
#endif
#define MAX_ACTIVATIONS (4096)
static OPUS_INLINE void vec_swish(float *y, const float *x, int N)
{
int i;
float tmp[MAX_ACTIVATIONS];
celt_assert(N <= MAX_ACTIVATIONS);
vec_sigmoid(tmp, x, N);
for (i=0;i<N;i++)
y[i] = x[i]*tmp[i];
}
static OPUS_INLINE float relu(float x)
{
return x < 0 ? 0 : x;
}
/*#define HIGH_ACCURACY */
void RTCD_SUF(compute_activation_)(float *output, const float *input, int N, int activation)
{
int i;
if (activation == ACTIVATION_SIGMOID) {
#ifdef HIGH_ACCURACY
for (int n=0; n<N; n++)
{
output[n] = 1.f / (1 + exp(-input[n]));
}
#else
vec_sigmoid(output, input, N);
#endif
} else if (activation == ACTIVATION_TANH) {
#ifdef HIGH_ACCURACY
for (int n=0; n<N; n++)
{
output[n] = tanh(input[n]);
}
#else
vec_tanh(output, input, N);
#endif
} else if (activation == ACTIVATION_SWISH) {
vec_swish(output, input, N);
} else if (activation == ACTIVATION_RELU) {
for (i=0;i<N;i++)
output[i] = relu(input[i]);
} else if (activation == ACTIVATION_SOFTMAX) {
#ifdef SOFTMAX_HACK
RNN_COPY(output, input, N);
/*for (i=0;i<N;i++)
output[i] = input[i];*/
#else
float sum = 0;
softmax(output, input, N);
for (i=0;i<N;i++) {
sum += output[i];
}
sum = 1.f/(sum+1e-30);
for (i=0;i<N;i++)
output[i] = sum*output[i];
#endif
} else {
celt_assert(activation == ACTIVATION_LINEAR);
if (input != output) {
for (i=0;i<N;i++)
output[i] = input[i];
}
}
}
void RTCD_SUF(compute_linear_) (const LinearLayer *linear, float *out, const float *in)
{
int i, M, N;
const float *bias;
celt_assert(in != out);
bias = linear->bias;
M = linear->nb_inputs;
N = linear->nb_outputs;
if (linear->float_weights != NULL) {
if (linear->weights_idx != NULL) sparse_sgemv8x4(out, linear->float_weights, linear->weights_idx, N, in);
else sgemv(out, linear->float_weights, N, M, N, in);
} else if (linear->weights != NULL) {
if (linear->weights_idx != NULL) sparse_cgemv8x4(out, linear->weights, linear->weights_idx, linear->scale, N, M, in);
else cgemv8x4(out, linear->weights, linear->scale, N, M, in);
/* Only use SU biases on for integer matrices on SU archs. */
#ifdef USE_SU_BIAS
bias = linear->subias;
#endif
}
else RNN_CLEAR(out, N);
if (bias != NULL) {
for (i=0;i<N;i++) out[i] += bias[i];
}
if (linear->diag) {
/* Diag is only used for GRU recurrent weights. */
celt_assert(3*M == N);
for (i=0;i<M;i++) {
out[i] += linear->diag[i]*in[i];
out[i+M] += linear->diag[i+M]*in[i];
out[i+2*M] += linear->diag[i+2*M]*in[i];
}
}
}
/* Computes non-padded convolution for input [ ksize1 x in_channels x (len2+ksize2) ],
kernel [ out_channels x in_channels x ksize1 x ksize2 ],
storing the output as [ out_channels x len2 ].
We assume that the output dimension along the ksize1 axis is 1,
i.e. processing one frame at a time. */
static void conv2d_float(float *out, const float *weights, int in_channels, int out_channels, int ktime, int kheight, const float *in, int height, int hstride)
{
int i;
int in_stride;
in_stride = height+kheight-1;
for (i=0;i<out_channels;i++) {
int m;
RNN_CLEAR(&out[i*hstride], height);
for (m=0;m<in_channels;m++) {
int t;
for (t=0;t<ktime;t++) {
int h;
for (h=0;h<kheight;h++) {
int j;
for (j=0;j<height;j++) {
out[i*hstride + j] += weights[i*in_channels*ktime*kheight + m*ktime*kheight + t*kheight + h] *
in[t*in_channels*in_stride + m*in_stride + j + h];
}
}
}
}
}
}
/* There's no intrinsics in this function (or the one above) because the gcc (and hopefully other compiler) auto-vectorizer is smart enough to
produce the right code by itself based on the compile flags. */
static void conv2d_3x3_float(float *out, const float *weights, int in_channels, int out_channels, const float *in, int height, int hstride)
{
int i;
int in_stride;
int kheight, ktime;
kheight = ktime = 3;
in_stride = height+kheight-1;
for (i=0;i<out_channels;i++) {
int m;
RNN_CLEAR(&out[i*hstride], height);
for (m=0;m<in_channels;m++) {
int j;
for (j=0;j<height;j++) {
/* Unrolled version of previous function -- compiler will figure out the indexing simplifications. */
out[i*hstride + j] += weights[i*in_channels*ktime*kheight + m*ktime*kheight + 0*kheight + 0]*in[0*in_channels*in_stride + m*in_stride + j + 0]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 0*kheight + 1]*in[0*in_channels*in_stride + m*in_stride + j + 1]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 0*kheight + 2]*in[0*in_channels*in_stride + m*in_stride + j + 2]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 1*kheight + 0]*in[1*in_channels*in_stride + m*in_stride + j + 0]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 1*kheight + 1]*in[1*in_channels*in_stride + m*in_stride + j + 1]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 1*kheight + 2]*in[1*in_channels*in_stride + m*in_stride + j + 2]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 2*kheight + 0]*in[2*in_channels*in_stride + m*in_stride + j + 0]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 2*kheight + 1]*in[2*in_channels*in_stride + m*in_stride + j + 1]
+ weights[i*in_channels*ktime*kheight + m*ktime*kheight + 2*kheight + 2]*in[2*in_channels*in_stride + m*in_stride + j + 2];
}
}
}
}
#define MAX_CONV2D_INPUTS 8192
void RTCD_SUF(compute_conv2d_)(const Conv2dLayer *conv, float *out, float *mem, const float *in, int height, int hstride, int activation)
{
int i;
const float *bias;
float in_buf[MAX_CONV2D_INPUTS];
int time_stride;
celt_assert(in != out);
time_stride = conv->in_channels*(height+conv->kheight-1);
celt_assert(conv->ktime*time_stride <= MAX_CONV2D_INPUTS);
RNN_COPY(in_buf, mem, (conv->ktime-1)*time_stride);
RNN_COPY(&in_buf[(conv->ktime-1)*time_stride], in, time_stride);
RNN_COPY(mem, &in_buf[time_stride], (conv->ktime-1)*time_stride);
bias = conv->bias;
if (conv->kheight == 3 && conv->ktime == 3)
conv2d_3x3_float(out, conv->float_weights, conv->in_channels, conv->out_channels, in_buf, height, hstride);
else
conv2d_float(out, conv->float_weights, conv->in_channels, conv->out_channels, conv->ktime, conv->kheight, in_buf, height, hstride);
if (bias != NULL) {
for (i=0;i<conv->out_channels;i++) {
int j;
for (j=0;j<height;j++) out[i*hstride+j] += bias[i];
}
}
for (i=0;i<conv->out_channels;i++) {
RTCD_SUF(compute_activation_)(&out[i*hstride], &out[i*hstride], height, activation);
}
}
#ifdef GCC_POP_OPTIONS
#pragma GCC pop_options
#endif
#endif

35
third_party/rnnoise/src/nnet_default.c vendored Normal file
View File

@@ -0,0 +1,35 @@
/* Copyright (c) 2018-2019 Mozilla
2023 Amazon */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#define RTCD_ARCH c
#include "nnet_arch.h"

159
third_party/rnnoise/src/opus_types.h vendored Normal file
View File

@@ -0,0 +1,159 @@
/* (C) COPYRIGHT 1994-2002 Xiph.Org Foundation */
/* Modified by Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* opus_types.h based on ogg_types.h from libogg */
/**
@file opus_types.h
@brief Opus reference implementation types
*/
#ifndef OPUS_TYPES_H
#define OPUS_TYPES_H
/* Use the real stdint.h if it's there (taken from Paul Hsieh's pstdint.h) */
#if (defined(__STDC__) && __STDC__ && defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L) || (defined(__GNUC__) && (defined(_STDINT_H) || defined(_STDINT_H_)) || defined (HAVE_STDINT_H))
#include <stdint.h>
typedef int16_t opus_int16;
typedef uint16_t opus_uint16;
typedef int32_t opus_int32;
typedef uint32_t opus_uint32;
#elif defined(_WIN32)
# if defined(__CYGWIN__)
# include <_G_config.h>
typedef _G_int32_t opus_int32;
typedef _G_uint32_t opus_uint32;
typedef _G_int16 opus_int16;
typedef _G_uint16 opus_uint16;
# elif defined(__MINGW32__)
typedef short opus_int16;
typedef unsigned short opus_uint16;
typedef int opus_int32;
typedef unsigned int opus_uint32;
# elif defined(__MWERKS__)
typedef int opus_int32;
typedef unsigned int opus_uint32;
typedef short opus_int16;
typedef unsigned short opus_uint16;
# else
/* MSVC/Borland */
typedef __int32 opus_int32;
typedef unsigned __int32 opus_uint32;
typedef __int16 opus_int16;
typedef unsigned __int16 opus_uint16;
# endif
#elif defined(__MACOS__)
# include <sys/types.h>
typedef SInt16 opus_int16;
typedef UInt16 opus_uint16;
typedef SInt32 opus_int32;
typedef UInt32 opus_uint32;
#elif (defined(__APPLE__) && defined(__MACH__)) /* MacOS X Framework build */
# include <sys/types.h>
typedef int16_t opus_int16;
typedef u_int16_t opus_uint16;
typedef int32_t opus_int32;
typedef u_int32_t opus_uint32;
#elif defined(__BEOS__)
/* Be */
# include <inttypes.h>
typedef int16 opus_int16;
typedef u_int16 opus_uint16;
typedef int32_t opus_int32;
typedef u_int32_t opus_uint32;
#elif defined (__EMX__)
/* OS/2 GCC */
typedef short opus_int16;
typedef unsigned short opus_uint16;
typedef int opus_int32;
typedef unsigned int opus_uint32;
#elif defined (DJGPP)
/* DJGPP */
typedef short opus_int16;
typedef unsigned short opus_uint16;
typedef int opus_int32;
typedef unsigned int opus_uint32;
#elif defined(R5900)
/* PS2 EE */
typedef int opus_int32;
typedef unsigned opus_uint32;
typedef short opus_int16;
typedef unsigned short opus_uint16;
#elif defined(__SYMBIAN32__)
/* Symbian GCC */
typedef signed short opus_int16;
typedef unsigned short opus_uint16;
typedef signed int opus_int32;
typedef unsigned int opus_uint32;
#elif defined(CONFIG_TI_C54X) || defined (CONFIG_TI_C55X)
typedef short opus_int16;
typedef unsigned short opus_uint16;
typedef long opus_int32;
typedef unsigned long opus_uint32;
#elif defined(CONFIG_TI_C6X)
typedef short opus_int16;
typedef unsigned short opus_uint16;
typedef int opus_int32;
typedef unsigned int opus_uint32;
#else
/* Give up, take a reasonable guess */
typedef short opus_int16;
typedef unsigned short opus_uint16;
typedef int opus_int32;
typedef unsigned int opus_uint32;
#endif
#define opus_int int /* used for counters etc; at least 16 bits */
#define opus_int64 long long
#define opus_int8 signed char
#define opus_uint unsigned int /* used for counters etc; at least 16 bits */
#define opus_uint64 unsigned long long
#define opus_uint8 unsigned char
#endif /* OPUS_TYPES_H */

237
third_party/rnnoise/src/parse_lpcnet_weights.c vendored Normal file
View File

@@ -0,0 +1,237 @@
/* Copyright (c) 2023 Amazon */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include <string.h>
#include <stdlib.h>
#include "nnet.h"
#define SPARSE_BLOCK_SIZE 32
static int parse_record(const void **data, int *len, WeightArray *array) {
WeightHead *h = (WeightHead *)*data;
if (*len < WEIGHT_BLOCK_SIZE) return -1;
if (h->block_size < h->size) return -1;
if (h->block_size > *len-WEIGHT_BLOCK_SIZE) return -1;
if (h->name[sizeof(h->name)-1] != 0) return -1;
if (h->size < 0) return -1;
array->name = h->name;
array->type = h->type;
array->size = h->size;
array->data = (void*)((unsigned char*)(*data)+WEIGHT_BLOCK_SIZE);
*data = (void*)((unsigned char*)*data + h->block_size+WEIGHT_BLOCK_SIZE);
*len -= h->block_size+WEIGHT_BLOCK_SIZE;
return array->size;
}
int parse_weights(WeightArray **list, const void *data, int len)
{
int nb_arrays=0;
int capacity=20;
*list = calloc(capacity*sizeof(WeightArray), 1);
while (len > 0) {
int ret;
WeightArray array = {NULL, 0, 0, 0};
ret = parse_record(&data, &len, &array);
if (ret > 0) {
if (nb_arrays+1 >= capacity) {
/* Make sure there's room for the ending NULL element too. */
capacity = capacity*3/2;
*list = realloc(*list, capacity*sizeof(WeightArray));
}
(*list)[nb_arrays++] = array;
} else {
free(*list);
*list = NULL;
return -1;
}
}
(*list)[nb_arrays].name=NULL;
return nb_arrays;
}
static const void *find_array_entry(const WeightArray *arrays, const char *name) {
while (arrays->name && strcmp(arrays->name, name) != 0) arrays++;
return arrays;
}
static const void *find_array_check(const WeightArray *arrays, const char *name, int size) {
const WeightArray *a = find_array_entry(arrays, name);
if (a->name && a->size == size) return a->data;
else return NULL;
}
static const void *opt_array_check(const WeightArray *arrays, const char *name, int size, int *error) {
const WeightArray *a = find_array_entry(arrays, name);
*error = (a->name != NULL && a->size != size);
if (a->name && a->size == size) return a->data;
else return NULL;
}
static const void *find_idx_check(const WeightArray *arrays, const char *name, int nb_in, int nb_out, int *total_blocks) {
int remain;
const int *idx;
const WeightArray *a = find_array_entry(arrays, name);
*total_blocks = 0;
if (a == NULL) return NULL;
idx = a->data;
remain = a->size/sizeof(int);
while (remain > 0) {
int nb_blocks;
int i;
nb_blocks = *idx++;
if (remain < nb_blocks+1) return NULL;
for (i=0;i<nb_blocks;i++) {
int pos = *idx++;
if (pos+3 >= nb_in || (pos&0x3)) return NULL;
}
nb_out -= 8;
remain -= nb_blocks+1;
*total_blocks += nb_blocks;
}
if (nb_out != 0) return NULL;
return a->data;
}
int linear_init(LinearLayer *layer, const WeightArray *arrays,
const char *bias,
const char *subias,
const char *weights,
const char *float_weights,
const char *weights_idx,
const char *diag,
const char *scale,
int nb_inputs,
int nb_outputs)
{
int err;
layer->bias = NULL;
layer->subias = NULL;
layer->weights = NULL;
layer->float_weights = NULL;
layer->weights_idx = NULL;
layer->diag = NULL;
layer->scale = NULL;
if (bias != NULL) {
if ((layer->bias = find_array_check(arrays, bias, nb_outputs*sizeof(layer->bias[0]))) == NULL) return 1;
}
if (subias != NULL) {
if ((layer->subias = find_array_check(arrays, subias, nb_outputs*sizeof(layer->subias[0]))) == NULL) return 1;
}
if (weights_idx != NULL) {
int total_blocks;
if ((layer->weights_idx = find_idx_check(arrays, weights_idx, nb_inputs, nb_outputs, &total_blocks)) == NULL) return 1;
if (weights != NULL) {
if ((layer->weights = find_array_check(arrays, weights, SPARSE_BLOCK_SIZE*total_blocks*sizeof(layer->weights[0]))) == NULL) return 1;
}
if (float_weights != NULL) {
layer->float_weights = opt_array_check(arrays, float_weights, SPARSE_BLOCK_SIZE*total_blocks*sizeof(layer->float_weights[0]), &err);
if (err) return 1;
}
} else {
if (weights != NULL) {
if ((layer->weights = find_array_check(arrays, weights, nb_inputs*nb_outputs*sizeof(layer->weights[0]))) == NULL) return 1;
}
if (float_weights != NULL) {
layer->float_weights = opt_array_check(arrays, float_weights, nb_inputs*nb_outputs*sizeof(layer->float_weights[0]), &err);
if (err) return 1;
}
}
if (diag != NULL) {
if ((layer->diag = find_array_check(arrays, diag, nb_outputs*sizeof(layer->diag[0]))) == NULL) return 1;
}
if (weights != NULL) {
if ((layer->scale = find_array_check(arrays, scale, nb_outputs*sizeof(layer->scale[0]))) == NULL) return 1;
}
layer->nb_inputs = nb_inputs;
layer->nb_outputs = nb_outputs;
return 0;
}
int conv2d_init(Conv2dLayer *layer, const WeightArray *arrays,
const char *bias,
const char *float_weights,
int in_channels,
int out_channels,
int ktime,
int kheight)
{
int err;
layer->bias = NULL;
layer->float_weights = NULL;
if (bias != NULL) {
if ((layer->bias = find_array_check(arrays, bias, out_channels*sizeof(layer->bias[0]))) == NULL) return 1;
}
if (float_weights != NULL) {
layer->float_weights = opt_array_check(arrays, float_weights, in_channels*out_channels*ktime*kheight*sizeof(layer->float_weights[0]), &err);
if (err) return 1;
}
layer->in_channels = in_channels;
layer->out_channels = out_channels;
layer->ktime = ktime;
layer->kheight = kheight;
return 0;
}
#if 0
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <sys/stat.h>
#include <stdio.h>
int main()
{
int fd;
void *data;
int len;
int nb_arrays;
int i;
WeightArray *list;
struct stat st;
const char *filename = "weights_blob.bin";
stat(filename, &st);
len = st.st_size;
fd = open(filename, O_RDONLY);
data = mmap(NULL, len, PROT_READ, MAP_SHARED, fd, 0);
printf("size is %d\n", len);
nb_arrays = parse_weights(&list, data, len);
for (i=0;i<nb_arrays;i++) {
printf("found %s: size %d\n", list[i].name, list[i].size);
}
printf("%p\n", list[i].name);
free(list);
munmap(data, len);
close(fd);
return 0;
}
#endif

528
third_party/rnnoise/src/pitch.c vendored Normal file
View File

@@ -0,0 +1,528 @@
/* Copyright (c) 2007-2008 CSIRO
Copyright (c) 2007-2009 Xiph.Org Foundation
Written by Jean-Marc Valin */
/**
@file pitch.c
@brief Pitch analysis
*/
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "pitch.h"
#include "common.h"
#include "denoise.h"
#include "celt_lpc.h"
#include "math.h"
static void find_best_pitch(opus_val32 *xcorr, opus_val16 *y, int len,
int max_pitch, int *best_pitch
#ifdef FIXED_POINT
, int yshift, opus_val32 maxcorr
#endif
)
{
int i, j;
opus_val32 Syy=1;
opus_val16 best_num[2];
opus_val32 best_den[2];
#ifdef FIXED_POINT
int xshift;
xshift = celt_ilog2(maxcorr)-14;
#endif
best_num[0] = -1;
best_num[1] = -1;
best_den[0] = 0;
best_den[1] = 0;
best_pitch[0] = 0;
best_pitch[1] = 1;
for (j=0;j<len;j++)
Syy = ADD32(Syy, SHR32(MULT16_16(y[j],y[j]), yshift));
for (i=0;i<max_pitch;i++)
{
if (xcorr[i]>0)
{
opus_val16 num;
opus_val32 xcorr16;
xcorr16 = EXTRACT16(VSHR32(xcorr[i], xshift));
#ifndef FIXED_POINT
/* Considering the range of xcorr16, this should avoid both underflows
and overflows (inf) when squaring xcorr16 */
xcorr16 *= 1e-12f;
#endif
num = MULT16_16_Q15(xcorr16,xcorr16);
if (MULT16_32_Q15(num,best_den[1]) > MULT16_32_Q15(best_num[1],Syy))
{
if (MULT16_32_Q15(num,best_den[0]) > MULT16_32_Q15(best_num[0],Syy))
{
best_num[1] = best_num[0];
best_den[1] = best_den[0];
best_pitch[1] = best_pitch[0];
best_num[0] = num;
best_den[0] = Syy;
best_pitch[0] = i;
} else {
best_num[1] = num;
best_den[1] = Syy;
best_pitch[1] = i;
}
}
}
Syy += SHR32(MULT16_16(y[i+len],y[i+len]),yshift) - SHR32(MULT16_16(y[i],y[i]),yshift);
Syy = MAX32(1, Syy);
}
}
static void celt_fir5(const opus_val16 *x,
const opus_val16 *num,
opus_val16 *y,
int N,
opus_val16 *mem)
{
int i;
opus_val16 num0, num1, num2, num3, num4;
opus_val32 mem0, mem1, mem2, mem3, mem4;
num0=num[0];
num1=num[1];
num2=num[2];
num3=num[3];
num4=num[4];
mem0=mem[0];
mem1=mem[1];
mem2=mem[2];
mem3=mem[3];
mem4=mem[4];
for (i=0;i<N;i++)
{
opus_val32 sum = SHL32(EXTEND32(x[i]), SIG_SHIFT);
sum = MAC16_16(sum,num0,mem0);
sum = MAC16_16(sum,num1,mem1);
sum = MAC16_16(sum,num2,mem2);
sum = MAC16_16(sum,num3,mem3);
sum = MAC16_16(sum,num4,mem4);
mem4 = mem3;
mem3 = mem2;
mem2 = mem1;
mem1 = mem0;
mem0 = x[i];
y[i] = ROUND16(sum, SIG_SHIFT);
}
mem[0]=mem0;
mem[1]=mem1;
mem[2]=mem2;
mem[3]=mem3;
mem[4]=mem4;
}
void rnn_pitch_downsample(celt_sig *x[], opus_val16 *x_lp,
int len, int C)
{
int i;
opus_val32 ac[5];
opus_val16 tmp=Q15ONE;
opus_val16 lpc[4], mem[5]={0,0,0,0,0};
opus_val16 lpc2[5];
opus_val16 c1 = QCONST16(.8f,15);
#ifdef FIXED_POINT
int shift;
opus_val32 maxabs = celt_maxabs32(x[0], len);
if (C==2)
{
opus_val32 maxabs_1 = celt_maxabs32(x[1], len);
maxabs = MAX32(maxabs, maxabs_1);
}
if (maxabs<1)
maxabs=1;
shift = celt_ilog2(maxabs)-10;
if (shift<0)
shift=0;
if (C==2)
shift++;
#endif
for (i=1;i<len>>1;i++)
x_lp[i] = SHR32(HALF32(HALF32(x[0][(2*i-1)]+x[0][(2*i+1)])+x[0][2*i]), shift);
x_lp[0] = SHR32(HALF32(HALF32(x[0][1])+x[0][0]), shift);
if (C==2)
{
for (i=1;i<len>>1;i++)
x_lp[i] += SHR32(HALF32(HALF32(x[1][(2*i-1)]+x[1][(2*i+1)])+x[1][2*i]), shift);
x_lp[0] += SHR32(HALF32(HALF32(x[1][1])+x[1][0]), shift);
}
rnn_autocorr(x_lp, ac, NULL, 0,
4, len>>1);
/* Noise floor -40 dB */
#ifdef FIXED_POINT
ac[0] += SHR32(ac[0],13);
#else
ac[0] *= 1.0001f;
#endif
/* Lag windowing */
for (i=1;i<=4;i++)
{
/*ac[i] *= exp(-.5*(2*M_PI*.002*i)*(2*M_PI*.002*i));*/
#ifdef FIXED_POINT
ac[i] -= MULT16_32_Q15(2*i*i, ac[i]);
#else
ac[i] -= ac[i]*(.008f*i)*(.008f*i);
#endif
}
rnn_lpc(lpc, ac, 4);
for (i=0;i<4;i++)
{
tmp = MULT16_16_Q15(QCONST16(.9f,15), tmp);
lpc[i] = MULT16_16_Q15(lpc[i], tmp);
}
/* Add a zero */
lpc2[0] = lpc[0] + QCONST16(.8f,SIG_SHIFT);
lpc2[1] = lpc[1] + MULT16_16_Q15(c1,lpc[0]);
lpc2[2] = lpc[2] + MULT16_16_Q15(c1,lpc[1]);
lpc2[3] = lpc[3] + MULT16_16_Q15(c1,lpc[2]);
lpc2[4] = MULT16_16_Q15(c1,lpc[3]);
celt_fir5(x_lp, lpc2, x_lp, len>>1, mem);
}
void rnn_pitch_xcorr(const opus_val16 *_x, const opus_val16 *_y,
opus_val32 *xcorr, int len, int max_pitch)
{
#if 0 /* This is a simple version of the pitch correlation that should work
well on DSPs like Blackfin and TI C5x/C6x */
int i, j;
#ifdef FIXED_POINT
opus_val32 maxcorr=1;
#endif
for (i=0;i<max_pitch;i++)
{
opus_val32 sum = 0;
for (j=0;j<len;j++)
sum = MAC16_16(sum, _x[j], _y[i+j]);
xcorr[i] = sum;
#ifdef FIXED_POINT
maxcorr = MAX32(maxcorr, sum);
#endif
}
#ifdef FIXED_POINT
return maxcorr;
#endif
#else /* Unrolled version of the pitch correlation -- runs faster on x86 and ARM */
int i;
/*The EDSP version requires that max_pitch is at least 1, and that _x is
32-bit aligned.
Since it's hard to put asserts in assembly, put them here.*/
#ifdef FIXED_POINT
opus_val32 maxcorr=1;
#endif
celt_assert(max_pitch>0);
celt_assert((((unsigned char *)_x-(unsigned char *)NULL)&3)==0);
for (i=0;i<max_pitch-3;i+=4)
{
opus_val32 sum[4]={0,0,0,0};
xcorr_kernel(_x, _y+i, sum, len);
xcorr[i]=sum[0];
xcorr[i+1]=sum[1];
xcorr[i+2]=sum[2];
xcorr[i+3]=sum[3];
#ifdef FIXED_POINT
sum[0] = MAX32(sum[0], sum[1]);
sum[2] = MAX32(sum[2], sum[3]);
sum[0] = MAX32(sum[0], sum[2]);
maxcorr = MAX32(maxcorr, sum[0]);
#endif
}
/* In case max_pitch isn't a multiple of 4, do non-unrolled version. */
for (;i<max_pitch;i++)
{
opus_val32 sum;
sum = celt_inner_prod(_x, _y+i, len);
xcorr[i] = sum;
#ifdef FIXED_POINT
maxcorr = MAX32(maxcorr, sum);
#endif
}
#ifdef FIXED_POINT
return maxcorr;
#endif
#endif
}
void rnn_pitch_search(const opus_val16 *x_lp, opus_val16 *y,
int len, int max_pitch, int *pitch)
{
int i, j;
int lag;
int best_pitch[2]={0,0};
#ifdef FIXED_POINT
opus_val32 maxcorr;
opus_val32 xmax, ymax;
int shift=0;
#endif
int offset;
opus_val16 x_lp4[PITCH_FRAME_SIZE>>2];
opus_val16 y_lp4[(PITCH_FRAME_SIZE+PITCH_MAX_PERIOD)>>2];
opus_val32 xcorr[PITCH_MAX_PERIOD>>1];
celt_assert(len <= PITCH_FRAME_SIZE);
celt_assert(max_pitch <= PITCH_MAX_PERIOD);
celt_assert(len>0);
celt_assert(max_pitch>0);
lag = len+max_pitch;
/* Downsample by 2 again */
for (j=0;j<len>>2;j++)
x_lp4[j] = x_lp[2*j];
for (j=0;j<lag>>2;j++)
y_lp4[j] = y[2*j];
#ifdef FIXED_POINT
xmax = celt_maxabs16(x_lp4, len>>2);
ymax = celt_maxabs16(y_lp4, lag>>2);
shift = celt_ilog2(MAX32(1, MAX32(xmax, ymax)))-11;
if (shift>0)
{
for (j=0;j<len>>2;j++)
x_lp4[j] = SHR16(x_lp4[j], shift);
for (j=0;j<lag>>2;j++)
y_lp4[j] = SHR16(y_lp4[j], shift);
/* Use double the shift for a MAC */
shift *= 2;
} else {
shift = 0;
}
#endif
/* Coarse search with 4x decimation */
#ifdef FIXED_POINT
maxcorr =
#endif
rnn_pitch_xcorr(x_lp4, y_lp4, xcorr, len>>2, max_pitch>>2);
find_best_pitch(xcorr, y_lp4, len>>2, max_pitch>>2, best_pitch
#ifdef FIXED_POINT
, 0, maxcorr
#endif
);
/* Finer search with 2x decimation */
#ifdef FIXED_POINT
maxcorr=1;
#endif
for (i=0;i<max_pitch>>1;i++)
{
opus_val32 sum;
xcorr[i] = 0;
if (abs(i-2*best_pitch[0])>2 && abs(i-2*best_pitch[1])>2)
continue;
#ifdef FIXED_POINT
sum = 0;
for (j=0;j<len>>1;j++)
sum += SHR32(MULT16_16(x_lp[j],y[i+j]), shift);
#else
sum = celt_inner_prod(x_lp, y+i, len>>1);
#endif
xcorr[i] = MAX32(-1, sum);
#ifdef FIXED_POINT
maxcorr = MAX32(maxcorr, sum);
#endif
}
find_best_pitch(xcorr, y, len>>1, max_pitch>>1, best_pitch
#ifdef FIXED_POINT
, shift+1, maxcorr
#endif
);
/* Refine by pseudo-interpolation */
if (best_pitch[0]>0 && best_pitch[0]<(max_pitch>>1)-1)
{
opus_val32 a, b, c;
a = xcorr[best_pitch[0]-1];
b = xcorr[best_pitch[0]];
c = xcorr[best_pitch[0]+1];
if ((c-a) > MULT16_32_Q15(QCONST16(.7f,15),b-a))
offset = 1;
else if ((a-c) > MULT16_32_Q15(QCONST16(.7f,15),b-c))
offset = -1;
else
offset = 0;
} else {
offset = 0;
}
*pitch = 2*best_pitch[0]-offset;
}
#ifdef FIXED_POINT
static opus_val16 compute_pitch_gain(opus_val32 xy, opus_val32 xx, opus_val32 yy)
{
opus_val32 x2y2;
int sx, sy, shift;
opus_val32 g;
opus_val16 den;
if (xy == 0 || xx == 0 || yy == 0)
return 0;
sx = celt_ilog2(xx)-14;
sy = celt_ilog2(yy)-14;
shift = sx + sy;
x2y2 = SHR32(MULT16_16(VSHR32(xx, sx), VSHR32(yy, sy)), 14);
if (shift & 1) {
if (x2y2 < 32768)
{
x2y2 <<= 1;
shift--;
} else {
x2y2 >>= 1;
shift++;
}
}
den = celt_rsqrt_norm(x2y2);
g = MULT16_32_Q15(den, xy);
g = VSHR32(g, (shift>>1)-1);
return EXTRACT16(MIN32(g, Q15ONE));
}
#else
static opus_val16 compute_pitch_gain(opus_val32 xy, opus_val32 xx, opus_val32 yy)
{
return xy/sqrt(1+xx*yy);
}
#endif
static const int second_check[16] = {0, 0, 3, 2, 3, 2, 5, 2, 3, 2, 3, 2, 5, 2, 3, 2};
opus_val16 rnn_remove_doubling(opus_val16 *x, int maxperiod, int minperiod,
int N, int *T0_, int prev_period, opus_val16 prev_gain)
{
int k, i, T, T0;
opus_val16 g, g0;
opus_val16 pg;
opus_val32 xy,xx,yy,xy2;
opus_val32 xcorr[3];
opus_val32 best_xy, best_yy;
int offset;
int minperiod0;
opus_val32 yy_lookup[PITCH_MAX_PERIOD+1];
celt_assert(maxperiod <= PITCH_MAX_PERIOD);
minperiod0 = minperiod;
maxperiod /= 2;
minperiod /= 2;
*T0_ /= 2;
prev_period /= 2;
N /= 2;
x += maxperiod;
if (*T0_>=maxperiod)
*T0_=maxperiod-1;
T = T0 = *T0_;
dual_inner_prod(x, x, x-T0, N, &xx, &xy);
yy_lookup[0] = xx;
yy=xx;
for (i=1;i<=maxperiod;i++)
{
yy = yy+MULT16_16(x[-i],x[-i])-MULT16_16(x[N-i],x[N-i]);
yy_lookup[i] = MAX32(0, yy);
}
yy = yy_lookup[T0];
best_xy = xy;
best_yy = yy;
g = g0 = compute_pitch_gain(xy, xx, yy);
/* Look for any pitch at T/k */
for (k=2;k<=15;k++)
{
int T1, T1b;
opus_val16 g1;
opus_val16 cont=0;
opus_val16 thresh;
T1 = (2*T0+k)/(2*k);
if (T1 < minperiod)
break;
/* Look for another strong correlation at T1b */
if (k==2)
{
if (T1+T0>maxperiod)
T1b = T0;
else
T1b = T0+T1;
} else
{
T1b = (2*second_check[k]*T0+k)/(2*k);
}
dual_inner_prod(x, &x[-T1], &x[-T1b], N, &xy, &xy2);
xy = HALF32(xy + xy2);
yy = HALF32(yy_lookup[T1] + yy_lookup[T1b]);
g1 = compute_pitch_gain(xy, xx, yy);
if (abs(T1-prev_period)<=1)
cont = prev_gain;
else if (abs(T1-prev_period)<=2 && 5*k*k < T0)
cont = HALF16(prev_gain);
else
cont = 0;
thresh = MAX16(QCONST16(.3f,15), MULT16_16_Q15(QCONST16(.7f,15),g0)-cont);
/* Bias against very high pitch (very short period) to avoid false-positives
due to short-term correlation */
if (T1<3*minperiod)
thresh = MAX16(QCONST16(.4f,15), MULT16_16_Q15(QCONST16(.85f,15),g0)-cont);
else if (T1<2*minperiod)
thresh = MAX16(QCONST16(.5f,15), MULT16_16_Q15(QCONST16(.9f,15),g0)-cont);
if (g1 > thresh)
{
best_xy = xy;
best_yy = yy;
T = T1;
g = g1;
}
}
best_xy = MAX32(0, best_xy);
if (best_yy <= best_xy)
pg = Q15ONE;
else
pg = best_xy/(best_yy+1);
for (k=0;k<3;k++)
xcorr[k] = celt_inner_prod(x, x-(T+k-1), N);
if ((xcorr[2]-xcorr[0]) > MULT16_32_Q15(QCONST16(.7f,15),xcorr[1]-xcorr[0]))
offset = 1;
else if ((xcorr[0]-xcorr[2]) > MULT16_32_Q15(QCONST16(.7f,15),xcorr[1]-xcorr[2]))
offset = -1;
else
offset = 0;
if (pg > g)
pg = g;
*T0_ = 2*T+offset;
if (*T0_<minperiod0)
*T0_=minperiod0;
return pg;
}

147
third_party/rnnoise/src/pitch.h vendored Normal file
View File

@@ -0,0 +1,147 @@
/* Copyright (c) 2007-2008 CSIRO
Copyright (c) 2007-2009 Xiph.Org Foundation
Written by Jean-Marc Valin */
/**
@file pitch.h
@brief Pitch analysis
*/
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef PITCH_H
#define PITCH_H
#include "arch.h"
void rnn_pitch_downsample(celt_sig *x[], opus_val16 *x_lp,
int len, int C);
void rnn_pitch_search(const opus_val16 *x_lp, opus_val16 *y,
int len, int max_pitch, int *pitch);
opus_val16 rnn_remove_doubling(opus_val16 *x, int maxperiod, int minperiod,
int N, int *T0, int prev_period, opus_val16 prev_gain);
/* OPT: This is the kernel you really want to optimize. It gets used a lot
by the prefilter and by the PLC. */
static OPUS_INLINE void xcorr_kernel(const opus_val16 * x, const opus_val16 * y, opus_val32 sum[4], int len)
{
int j;
opus_val16 y_0, y_1, y_2, y_3;
celt_assert(len>=3);
y_3=0; /* gcc doesn't realize that y_3 can't be used uninitialized */
y_0=*y++;
y_1=*y++;
y_2=*y++;
for (j=0;j<len-3;j+=4)
{
opus_val16 tmp;
tmp = *x++;
y_3=*y++;
sum[0] = MAC16_16(sum[0],tmp,y_0);
sum[1] = MAC16_16(sum[1],tmp,y_1);
sum[2] = MAC16_16(sum[2],tmp,y_2);
sum[3] = MAC16_16(sum[3],tmp,y_3);
tmp=*x++;
y_0=*y++;
sum[0] = MAC16_16(sum[0],tmp,y_1);
sum[1] = MAC16_16(sum[1],tmp,y_2);
sum[2] = MAC16_16(sum[2],tmp,y_3);
sum[3] = MAC16_16(sum[3],tmp,y_0);
tmp=*x++;
y_1=*y++;
sum[0] = MAC16_16(sum[0],tmp,y_2);
sum[1] = MAC16_16(sum[1],tmp,y_3);
sum[2] = MAC16_16(sum[2],tmp,y_0);
sum[3] = MAC16_16(sum[3],tmp,y_1);
tmp=*x++;
y_2=*y++;
sum[0] = MAC16_16(sum[0],tmp,y_3);
sum[1] = MAC16_16(sum[1],tmp,y_0);
sum[2] = MAC16_16(sum[2],tmp,y_1);
sum[3] = MAC16_16(sum[3],tmp,y_2);
}
if (j++<len)
{
opus_val16 tmp = *x++;
y_3=*y++;
sum[0] = MAC16_16(sum[0],tmp,y_0);
sum[1] = MAC16_16(sum[1],tmp,y_1);
sum[2] = MAC16_16(sum[2],tmp,y_2);
sum[3] = MAC16_16(sum[3],tmp,y_3);
}
if (j++<len)
{
opus_val16 tmp=*x++;
y_0=*y++;
sum[0] = MAC16_16(sum[0],tmp,y_1);
sum[1] = MAC16_16(sum[1],tmp,y_2);
sum[2] = MAC16_16(sum[2],tmp,y_3);
sum[3] = MAC16_16(sum[3],tmp,y_0);
}
if (j<len)
{
opus_val16 tmp=*x++;
y_1=*y++;
sum[0] = MAC16_16(sum[0],tmp,y_2);
sum[1] = MAC16_16(sum[1],tmp,y_3);
sum[2] = MAC16_16(sum[2],tmp,y_0);
sum[3] = MAC16_16(sum[3],tmp,y_1);
}
}
static OPUS_INLINE void dual_inner_prod(const opus_val16 *x, const opus_val16 *y01, const opus_val16 *y02,
int N, opus_val32 *xy1, opus_val32 *xy2)
{
int i;
opus_val32 xy01=0;
opus_val32 xy02=0;
for (i=0;i<N;i++)
{
xy01 = MAC16_16(xy01, x[i], y01[i]);
xy02 = MAC16_16(xy02, x[i], y02[i]);
}
*xy1 = xy01;
*xy2 = xy02;
}
/*We make sure a C version is always available for cases where the overhead of
vectorization and passing around an arch flag aren't worth it.*/
static OPUS_INLINE opus_val32 celt_inner_prod(const opus_val16 *x,
const opus_val16 *y, int N)
{
int i;
opus_val32 xy=0;
for (i=0;i<N;i++)
xy = MAC16_16(xy, x[i], y[i]);
return xy;
}
void rnn_pitch_xcorr(const opus_val16 *_x, const opus_val16 *_y,
opus_val32 *xcorr, int len, int max_pitch);
#endif

60
third_party/rnnoise/src/rnn.c vendored Normal file
View File

@@ -0,0 +1,60 @@
/* Copyright (c) 2008-2011 Octasic Inc.
2012-2017 Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include <math.h>
#include "opus_types.h"
#include "common.h"
#include "arch.h"
#include "rnn.h"
#include "rnnoise_data.h"
#include <stdio.h>
#define INPUT_SIZE 42
void compute_rnn(const RNNoise *model, RNNState *rnn, float *gains, float *vad, const float *input, int arch) {
float tmp[MAX_NEURONS];
float cat[CONV2_OUT_SIZE + GRU1_OUT_SIZE + GRU2_OUT_SIZE + GRU3_OUT_SIZE];
/*for (int i=0;i<INPUT_SIZE;i++) printf("%f ", input[i]);printf("\n");*/
compute_generic_conv1d(&model->conv1, tmp, rnn->conv1_state, input, CONV1_IN_SIZE, ACTIVATION_TANH, arch);
compute_generic_conv1d(&model->conv2, cat, rnn->conv2_state, tmp, CONV2_IN_SIZE, ACTIVATION_TANH, arch);
compute_generic_gru(&model->gru1_input, &model->gru1_recurrent, rnn->gru1_state, cat, arch);
compute_generic_gru(&model->gru2_input, &model->gru2_recurrent, rnn->gru2_state, rnn->gru1_state, arch);
compute_generic_gru(&model->gru3_input, &model->gru3_recurrent, rnn->gru3_state, rnn->gru2_state, arch);
RNN_COPY(&cat[CONV2_OUT_SIZE], rnn->gru1_state, GRU1_OUT_SIZE);
RNN_COPY(&cat[CONV2_OUT_SIZE+GRU1_OUT_SIZE], rnn->gru2_state, GRU2_OUT_SIZE);
RNN_COPY(&cat[CONV2_OUT_SIZE+GRU1_OUT_SIZE+GRU2_OUT_SIZE], rnn->gru3_state, GRU3_OUT_SIZE);
compute_generic_dense(&model->dense_out, gains, cat, ACTIVATION_SIGMOID, arch);
compute_generic_dense(&model->vad_dense, vad, cat, ACTIVATION_SIGMOID, arch);
/*for (int i=0;i<22;i++) printf("%f ", gains[i]);printf("\n");*/
/*printf("%f\n", *vad);*/
}

49
third_party/rnnoise/src/rnn.h vendored Normal file
View File

@@ -0,0 +1,49 @@
/* Copyright (c) 2017 Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef RNN_H_
#define RNN_H_
#include "rnnoise.h"
#include "rnnoise_data.h"
#include "opus_types.h"
#define WEIGHTS_SCALE (1.f/256)
#define MAX_NEURONS 1024
typedef struct {
float conv1_state[CONV1_STATE_SIZE];
float conv2_state[CONV2_STATE_SIZE];
float gru1_state[GRU1_STATE_SIZE];
float gru2_state[GRU2_STATE_SIZE];
float gru3_state[GRU3_STATE_SIZE];
} RNNState;
void compute_rnn(const RNNoise *model, RNNState *rnn, float *gains, float *vad, const float *input, int arch);
#endif /* RNN_H_ */

373966
third_party/rnnoise/src/rnnoise_data.c vendored Normal file
View File

File diff suppressed because it is too large Load Diff

55
third_party/rnnoise/src/rnnoise_data.h vendored Normal file
View File

@@ -0,0 +1,55 @@
#ifndef RNNOISE_DATA_H
#define RNNOISE_DATA_H
#include "nnet.h"
#define CONV1_OUT_SIZE 128
#define CONV1_IN_SIZE 65
#define CONV1_STATE_SIZE (65 * (2))
#define CONV1_DELAY 1
#define CONV2_OUT_SIZE 384
#define CONV2_IN_SIZE 128
#define CONV2_STATE_SIZE (128 * (2))
#define CONV2_DELAY 1
#define GRU1_OUT_SIZE 384
#define GRU1_STATE_SIZE 384
#define GRU2_OUT_SIZE 384
#define GRU2_STATE_SIZE 384
#define GRU3_OUT_SIZE 384
#define GRU3_STATE_SIZE 384
#define DENSE_OUT_OUT_SIZE 32
#define VAD_DENSE_OUT_SIZE 1
typedef struct {
LinearLayer conv1;
LinearLayer conv2;
LinearLayer gru1_input;
LinearLayer gru1_recurrent;
LinearLayer gru2_input;
LinearLayer gru2_recurrent;
LinearLayer gru3_input;
LinearLayer gru3_recurrent;
LinearLayer dense_out;
LinearLayer vad_dense;
} RNNoise;
int init_rnnoise(RNNoise *model, const WeightArray *arrays);
#endif /* RNNOISE_DATA_H */

874
third_party/rnnoise/src/rnnoise_tables.c vendored Normal file
View File

@@ -0,0 +1,874 @@
/* The contents of this file was automatically generated by dump_rnnoise_tables.c*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "kiss_fft.h"
static const arch_fft_state arch_fft = {0, NULL};
static const opus_int32 fft_bitrev[960] = {
0, 192, 384, 576, 768, 64, 256, 448, 640, 832, 128, 320, 512, 704, 896,
16, 208, 400, 592, 784, 80, 272, 464, 656, 848, 144, 336, 528, 720, 912,
32, 224, 416, 608, 800, 96, 288, 480, 672, 864, 160, 352, 544, 736, 928,
48, 240, 432, 624, 816, 112, 304, 496, 688, 880, 176, 368, 560, 752, 944,
4, 196, 388, 580, 772, 68, 260, 452, 644, 836, 132, 324, 516, 708, 900,
20, 212, 404, 596, 788, 84, 276, 468, 660, 852, 148, 340, 532, 724, 916,
36, 228, 420, 612, 804, 100, 292, 484, 676, 868, 164, 356, 548, 740, 932,
52, 244, 436, 628, 820, 116, 308, 500, 692, 884, 180, 372, 564, 756, 948,
8, 200, 392, 584, 776, 72, 264, 456, 648, 840, 136, 328, 520, 712, 904,
24, 216, 408, 600, 792, 88, 280, 472, 664, 856, 152, 344, 536, 728, 920,
40, 232, 424, 616, 808, 104, 296, 488, 680, 872, 168, 360, 552, 744, 936,
56, 248, 440, 632, 824, 120, 312, 504, 696, 888, 184, 376, 568, 760, 952,
12, 204, 396, 588, 780, 76, 268, 460, 652, 844, 140, 332, 524, 716, 908,
28, 220, 412, 604, 796, 92, 284, 476, 668, 860, 156, 348, 540, 732, 924,
44, 236, 428, 620, 812, 108, 300, 492, 684, 876, 172, 364, 556, 748, 940,
60, 252, 444, 636, 828, 124, 316, 508, 700, 892, 188, 380, 572, 764, 956,
1, 193, 385, 577, 769, 65, 257, 449, 641, 833, 129, 321, 513, 705, 897,
17, 209, 401, 593, 785, 81, 273, 465, 657, 849, 145, 337, 529, 721, 913,
33, 225, 417, 609, 801, 97, 289, 481, 673, 865, 161, 353, 545, 737, 929,
49, 241, 433, 625, 817, 113, 305, 497, 689, 881, 177, 369, 561, 753, 945,
5, 197, 389, 581, 773, 69, 261, 453, 645, 837, 133, 325, 517, 709, 901,
21, 213, 405, 597, 789, 85, 277, 469, 661, 853, 149, 341, 533, 725, 917,
37, 229, 421, 613, 805, 101, 293, 485, 677, 869, 165, 357, 549, 741, 933,
53, 245, 437, 629, 821, 117, 309, 501, 693, 885, 181, 373, 565, 757, 949,
9, 201, 393, 585, 777, 73, 265, 457, 649, 841, 137, 329, 521, 713, 905,
25, 217, 409, 601, 793, 89, 281, 473, 665, 857, 153, 345, 537, 729, 921,
41, 233, 425, 617, 809, 105, 297, 489, 681, 873, 169, 361, 553, 745, 937,
57, 249, 441, 633, 825, 121, 313, 505, 697, 889, 185, 377, 569, 761, 953,
13, 205, 397, 589, 781, 77, 269, 461, 653, 845, 141, 333, 525, 717, 909,
29, 221, 413, 605, 797, 93, 285, 477, 669, 861, 157, 349, 541, 733, 925,
45, 237, 429, 621, 813, 109, 301, 493, 685, 877, 173, 365, 557, 749, 941,
61, 253, 445, 637, 829, 125, 317, 509, 701, 893, 189, 381, 573, 765, 957,
2, 194, 386, 578, 770, 66, 258, 450, 642, 834, 130, 322, 514, 706, 898,
18, 210, 402, 594, 786, 82, 274, 466, 658, 850, 146, 338, 530, 722, 914,
34, 226, 418, 610, 802, 98, 290, 482, 674, 866, 162, 354, 546, 738, 930,
50, 242, 434, 626, 818, 114, 306, 498, 690, 882, 178, 370, 562, 754, 946,
6, 198, 390, 582, 774, 70, 262, 454, 646, 838, 134, 326, 518, 710, 902,
22, 214, 406, 598, 790, 86, 278, 470, 662, 854, 150, 342, 534, 726, 918,
38, 230, 422, 614, 806, 102, 294, 486, 678, 870, 166, 358, 550, 742, 934,
54, 246, 438, 630, 822, 118, 310, 502, 694, 886, 182, 374, 566, 758, 950,
10, 202, 394, 586, 778, 74, 266, 458, 650, 842, 138, 330, 522, 714, 906,
26, 218, 410, 602, 794, 90, 282, 474, 666, 858, 154, 346, 538, 730, 922,
42, 234, 426, 618, 810, 106, 298, 490, 682, 874, 170, 362, 554, 746, 938,
58, 250, 442, 634, 826, 122, 314, 506, 698, 890, 186, 378, 570, 762, 954,
14, 206, 398, 590, 782, 78, 270, 462, 654, 846, 142, 334, 526, 718, 910,
30, 222, 414, 606, 798, 94, 286, 478, 670, 862, 158, 350, 542, 734, 926,
46, 238, 430, 622, 814, 110, 302, 494, 686, 878, 174, 366, 558, 750, 942,
62, 254, 446, 638, 830, 126, 318, 510, 702, 894, 190, 382, 574, 766, 958,
3, 195, 387, 579, 771, 67, 259, 451, 643, 835, 131, 323, 515, 707, 899,
19, 211, 403, 595, 787, 83, 275, 467, 659, 851, 147, 339, 531, 723, 915,
35, 227, 419, 611, 803, 99, 291, 483, 675, 867, 163, 355, 547, 739, 931,
51, 243, 435, 627, 819, 115, 307, 499, 691, 883, 179, 371, 563, 755, 947,
7, 199, 391, 583, 775, 71, 263, 455, 647, 839, 135, 327, 519, 711, 903,
23, 215, 407, 599, 791, 87, 279, 471, 663, 855, 151, 343, 535, 727, 919,
39, 231, 423, 615, 807, 103, 295, 487, 679, 871, 167, 359, 551, 743, 935,
55, 247, 439, 631, 823, 119, 311, 503, 695, 887, 183, 375, 567, 759, 951,
11, 203, 395, 587, 779, 75, 267, 459, 651, 843, 139, 331, 523, 715, 907,
27, 219, 411, 603, 795, 91, 283, 475, 667, 859, 155, 347, 539, 731, 923,
43, 235, 427, 619, 811, 107, 299, 491, 683, 875, 171, 363, 555, 747, 939,
59, 251, 443, 635, 827, 123, 315, 507, 699, 891, 187, 379, 571, 763, 955,
15, 207, 399, 591, 783, 79, 271, 463, 655, 847, 143, 335, 527, 719, 911,
31, 223, 415, 607, 799, 95, 287, 479, 671, 863, 159, 351, 543, 735, 927,
47, 239, 431, 623, 815, 111, 303, 495, 687, 879, 175, 367, 559, 751, 943,
63, 255, 447, 639, 831, 127, 319, 511, 703, 895, 191, 383, 575, 767, 959,
};
static const kiss_twiddle_cpx fft_twiddles[960] = {
{1.00000000f, -0.00000000f}, {0.999978602f, -0.00654493785f},
{0.999914348f, -0.0130895954f}, {0.999807239f, -0.0196336918f},
{0.999657333f, -0.0261769481f}, {0.999464571f, -0.0327190831f},
{0.999229014f, -0.0392598175f}, {0.998950660f, -0.0457988679f},
{0.998629510f, -0.0523359552f}, {0.998265624f, -0.0588708036f},
{0.997858942f, -0.0654031262f}, {0.997409463f, -0.0719326511f},
{0.996917307f, -0.0784590989f}, {0.996382475f, -0.0849821791f},
{0.995804906f, -0.0915016159f}, {0.995184720f, -0.0980171412f},
{0.994521916f, -0.104528464f}, {0.993816435f, -0.111035310f},
{0.993068457f, -0.117537394f}, {0.992277920f, -0.124034449f},
{0.991444886f, -0.130526185f}, {0.990569353f, -0.137012348f},
{0.989651382f, -0.143492624f}, {0.988691032f, -0.149966761f},
{0.987688363f, -0.156434461f}, {0.986643314f, -0.162895471f},
{0.985556066f, -0.169349506f}, {0.984426558f, -0.175796285f},
{0.983254910f, -0.182235524f}, {0.982041121f, -0.188666970f},
{0.980785251f, -0.195090324f}, {0.979487419f, -0.201505318f},
{0.978147626f, -0.207911685f}, {0.976765871f, -0.214309156f},
{0.975342333f, -0.220697433f}, {0.973876953f, -0.227076262f},
{0.972369909f, -0.233445361f}, {0.970821202f, -0.239804462f},
{0.969230890f, -0.246153295f}, {0.967599094f, -0.252491564f},
{0.965925813f, -0.258819044f}, {0.964211166f, -0.265135437f},
{0.962455213f, -0.271440446f}, {0.960658073f, -0.277733833f},
{0.958819747f, -0.284015357f}, {0.956940353f, -0.290284663f},
{0.955019951f, -0.296541572f}, {0.953058660f, -0.302785784f},
{0.951056540f, -0.309017003f}, {0.949013650f, -0.315234989f},
{0.946930110f, -0.321439475f}, {0.944806039f, -0.327630192f},
{0.942641497f, -0.333806872f}, {0.940436542f, -0.339969248f},
{0.938191354f, -0.346117049f}, {0.935905933f, -0.352250040f},
{0.933580399f, -0.358367950f}, {0.931214929f, -0.364470512f},
{0.928809524f, -0.370557427f}, {0.926364362f, -0.376628488f},
{0.923879504f, -0.382683426f}, {0.921355128f, -0.388721973f},
{0.918791234f, -0.394743860f}, {0.916187942f, -0.400748819f},
{0.913545430f, -0.406736642f}, {0.910863817f, -0.412707031f},
{0.908143163f, -0.418659747f}, {0.905383646f, -0.424594522f},
{0.902585268f, -0.430511087f}, {0.899748266f, -0.436409235f},
{0.896872759f, -0.442288697f}, {0.893958807f, -0.448149204f},
{0.891006529f, -0.453990489f}, {0.888016105f, -0.459812373f},
{0.884987652f, -0.465614527f}, {0.881921291f, -0.471396744f},
{0.878817141f, -0.477158755f}, {0.875675321f, -0.482900351f},
{0.872496009f, -0.488621235f}, {0.869279325f, -0.494321197f},
{0.866025388f, -0.500000000f}, {0.862734377f, -0.505657375f},
{0.859406412f, -0.511293113f}, {0.856041610f, -0.516906917f},
{0.852640152f, -0.522498548f}, {0.849202156f, -0.528067827f},
{0.845727801f, -0.533614516f}, {0.842217207f, -0.539138317f},
{0.838670552f, -0.544639051f}, {0.835087955f, -0.550116420f},
{0.831469595f, -0.555570245f}, {0.827815652f, -0.561000228f},
{0.824126184f, -0.566406250f}, {0.820401430f, -0.571787953f},
{0.816641569f, -0.577145219f}, {0.812846661f, -0.582477689f},
{0.809017003f, -0.587785244f}, {0.805152655f, -0.593067646f},
{0.801253796f, -0.598324597f}, {0.797320664f, -0.603555918f},
{0.793353319f, -0.608761430f}, {0.789352059f, -0.613940835f},
{0.785316944f, -0.619093955f}, {0.781248152f, -0.624220550f},
{0.777145982f, -0.629320383f}, {0.773010433f, -0.634393275f},
{0.768841803f, -0.639438987f}, {0.764640272f, -0.644457340f},
{0.760405958f, -0.649448037f}, {0.756139100f, -0.654410958f},
{0.751839817f, -0.659345806f}, {0.747508347f, -0.664252460f},
{0.743144810f, -0.669130623f}, {0.738749504f, -0.673980117f},
{0.734322488f, -0.678800762f}, {0.729864061f, -0.683592319f},
{0.725374401f, -0.688354552f}, {0.720853567f, -0.693087339f},
{0.716301918f, -0.697790444f}, {0.711719632f, -0.702463686f},
{0.707106769f, -0.707106769f}, {0.702463686f, -0.711719632f},
{0.697790444f, -0.716301918f}, {0.693087339f, -0.720853567f},
{0.688354552f, -0.725374401f}, {0.683592319f, -0.729864061f},
{0.678800762f, -0.734322488f}, {0.673980117f, -0.738749504f},
{0.669130623f, -0.743144810f}, {0.664252460f, -0.747508347f},
{0.659345806f, -0.751839817f}, {0.654410958f, -0.756139100f},
{0.649448037f, -0.760405958f}, {0.644457340f, -0.764640272f},
{0.639438987f, -0.768841803f}, {0.634393275f, -0.773010433f},
{0.629320383f, -0.777145982f}, {0.624220550f, -0.781248152f},
{0.619093955f, -0.785316944f}, {0.613940835f, -0.789352059f},
{0.608761430f, -0.793353319f}, {0.603555918f, -0.797320664f},
{0.598324597f, -0.801253796f}, {0.593067646f, -0.805152655f},
{0.587785244f, -0.809017003f}, {0.582477689f, -0.812846661f},
{0.577145219f, -0.816641569f}, {0.571787953f, -0.820401430f},
{0.566406250f, -0.824126184f}, {0.561000228f, -0.827815652f},
{0.555570245f, -0.831469595f}, {0.550116420f, -0.835087955f},
{0.544639051f, -0.838670552f}, {0.539138317f, -0.842217207f},
{0.533614516f, -0.845727801f}, {0.528067827f, -0.849202156f},
{0.522498548f, -0.852640152f}, {0.516906917f, -0.856041610f},
{0.511293113f, -0.859406412f}, {0.505657375f, -0.862734377f},
{0.500000000f, -0.866025388f}, {0.494321197f, -0.869279325f},
{0.488621235f, -0.872496009f}, {0.482900351f, -0.875675321f},
{0.477158755f, -0.878817141f}, {0.471396744f, -0.881921291f},
{0.465614527f, -0.884987652f}, {0.459812373f, -0.888016105f},
{0.453990489f, -0.891006529f}, {0.448149204f, -0.893958807f},
{0.442288697f, -0.896872759f}, {0.436409235f, -0.899748266f},
{0.430511087f, -0.902585268f}, {0.424594522f, -0.905383646f},
{0.418659747f, -0.908143163f}, {0.412707031f, -0.910863817f},
{0.406736642f, -0.913545430f}, {0.400748819f, -0.916187942f},
{0.394743860f, -0.918791234f}, {0.388721973f, -0.921355128f},
{0.382683426f, -0.923879504f}, {0.376628488f, -0.926364362f},
{0.370557427f, -0.928809524f}, {0.364470512f, -0.931214929f},
{0.358367950f, -0.933580399f}, {0.352250040f, -0.935905933f},
{0.346117049f, -0.938191354f}, {0.339969248f, -0.940436542f},
{0.333806872f, -0.942641497f}, {0.327630192f, -0.944806039f},
{0.321439475f, -0.946930110f}, {0.315234989f, -0.949013650f},
{0.309017003f, -0.951056540f}, {0.302785784f, -0.953058660f},
{0.296541572f, -0.955019951f}, {0.290284663f, -0.956940353f},
{0.284015357f, -0.958819747f}, {0.277733833f, -0.960658073f},
{0.271440446f, -0.962455213f}, {0.265135437f, -0.964211166f},
{0.258819044f, -0.965925813f}, {0.252491564f, -0.967599094f},
{0.246153295f, -0.969230890f}, {0.239804462f, -0.970821202f},
{0.233445361f, -0.972369909f}, {0.227076262f, -0.973876953f},
{0.220697433f, -0.975342333f}, {0.214309156f, -0.976765871f},
{0.207911685f, -0.978147626f}, {0.201505318f, -0.979487419f},
{0.195090324f, -0.980785251f}, {0.188666970f, -0.982041121f},
{0.182235524f, -0.983254910f}, {0.175796285f, -0.984426558f},
{0.169349506f, -0.985556066f}, {0.162895471f, -0.986643314f},
{0.156434461f, -0.987688363f}, {0.149966761f, -0.988691032f},
{0.143492624f, -0.989651382f}, {0.137012348f, -0.990569353f},
{0.130526185f, -0.991444886f}, {0.124034449f, -0.992277920f},
{0.117537394f, -0.993068457f}, {0.111035310f, -0.993816435f},
{0.104528464f, -0.994521916f}, {0.0980171412f, -0.995184720f},
{0.0915016159f, -0.995804906f}, {0.0849821791f, -0.996382475f},
{0.0784590989f, -0.996917307f}, {0.0719326511f, -0.997409463f},
{0.0654031262f, -0.997858942f}, {0.0588708036f, -0.998265624f},
{0.0523359552f, -0.998629510f}, {0.0457988679f, -0.998950660f},
{0.0392598175f, -0.999229014f}, {0.0327190831f, -0.999464571f},
{0.0261769481f, -0.999657333f}, {0.0196336918f, -0.999807239f},
{0.0130895954f, -0.999914348f}, {0.00654493785f, -0.999978602f},
{6.12323426e-17f, -1.00000000f}, {-0.00654493785f, -0.999978602f},
{-0.0130895954f, -0.999914348f}, {-0.0196336918f, -0.999807239f},
{-0.0261769481f, -0.999657333f}, {-0.0327190831f, -0.999464571f},
{-0.0392598175f, -0.999229014f}, {-0.0457988679f, -0.998950660f},
{-0.0523359552f, -0.998629510f}, {-0.0588708036f, -0.998265624f},
{-0.0654031262f, -0.997858942f}, {-0.0719326511f, -0.997409463f},
{-0.0784590989f, -0.996917307f}, {-0.0849821791f, -0.996382475f},
{-0.0915016159f, -0.995804906f}, {-0.0980171412f, -0.995184720f},
{-0.104528464f, -0.994521916f}, {-0.111035310f, -0.993816435f},
{-0.117537394f, -0.993068457f}, {-0.124034449f, -0.992277920f},
{-0.130526185f, -0.991444886f}, {-0.137012348f, -0.990569353f},
{-0.143492624f, -0.989651382f}, {-0.149966761f, -0.988691032f},
{-0.156434461f, -0.987688363f}, {-0.162895471f, -0.986643314f},
{-0.169349506f, -0.985556066f}, {-0.175796285f, -0.984426558f},
{-0.182235524f, -0.983254910f}, {-0.188666970f, -0.982041121f},
{-0.195090324f, -0.980785251f}, {-0.201505318f, -0.979487419f},
{-0.207911685f, -0.978147626f}, {-0.214309156f, -0.976765871f},
{-0.220697433f, -0.975342333f}, {-0.227076262f, -0.973876953f},
{-0.233445361f, -0.972369909f}, {-0.239804462f, -0.970821202f},
{-0.246153295f, -0.969230890f}, {-0.252491564f, -0.967599094f},
{-0.258819044f, -0.965925813f}, {-0.265135437f, -0.964211166f},
{-0.271440446f, -0.962455213f}, {-0.277733833f, -0.960658073f},
{-0.284015357f, -0.958819747f}, {-0.290284663f, -0.956940353f},
{-0.296541572f, -0.955019951f}, {-0.302785784f, -0.953058660f},
{-0.309017003f, -0.951056540f}, {-0.315234989f, -0.949013650f},
{-0.321439475f, -0.946930110f}, {-0.327630192f, -0.944806039f},
{-0.333806872f, -0.942641497f}, {-0.339969248f, -0.940436542f},
{-0.346117049f, -0.938191354f}, {-0.352250040f, -0.935905933f},
{-0.358367950f, -0.933580399f}, {-0.364470512f, -0.931214929f},
{-0.370557427f, -0.928809524f}, {-0.376628488f, -0.926364362f},
{-0.382683426f, -0.923879504f}, {-0.388721973f, -0.921355128f},
{-0.394743860f, -0.918791234f}, {-0.400748819f, -0.916187942f},
{-0.406736642f, -0.913545430f}, {-0.412707031f, -0.910863817f},
{-0.418659747f, -0.908143163f}, {-0.424594522f, -0.905383646f},
{-0.430511087f, -0.902585268f}, {-0.436409235f, -0.899748266f},
{-0.442288697f, -0.896872759f}, {-0.448149204f, -0.893958807f},
{-0.453990489f, -0.891006529f}, {-0.459812373f, -0.888016105f},
{-0.465614527f, -0.884987652f}, {-0.471396744f, -0.881921291f},
{-0.477158755f, -0.878817141f}, {-0.482900351f, -0.875675321f},
{-0.488621235f, -0.872496009f}, {-0.494321197f, -0.869279325f},
{-0.500000000f, -0.866025388f}, {-0.505657375f, -0.862734377f},
{-0.511293113f, -0.859406412f}, {-0.516906917f, -0.856041610f},
{-0.522498548f, -0.852640152f}, {-0.528067827f, -0.849202156f},
{-0.533614516f, -0.845727801f}, {-0.539138317f, -0.842217207f},
{-0.544639051f, -0.838670552f}, {-0.550116420f, -0.835087955f},
{-0.555570245f, -0.831469595f}, {-0.561000228f, -0.827815652f},
{-0.566406250f, -0.824126184f}, {-0.571787953f, -0.820401430f},
{-0.577145219f, -0.816641569f}, {-0.582477689f, -0.812846661f},
{-0.587785244f, -0.809017003f}, {-0.593067646f, -0.805152655f},
{-0.598324597f, -0.801253796f}, {-0.603555918f, -0.797320664f},
{-0.608761430f, -0.793353319f}, {-0.613940835f, -0.789352059f},
{-0.619093955f, -0.785316944f}, {-0.624220550f, -0.781248152f},
{-0.629320383f, -0.777145982f}, {-0.634393275f, -0.773010433f},
{-0.639438987f, -0.768841803f}, {-0.644457340f, -0.764640272f},
{-0.649448037f, -0.760405958f}, {-0.654410958f, -0.756139100f},
{-0.659345806f, -0.751839817f}, {-0.664252460f, -0.747508347f},
{-0.669130623f, -0.743144810f}, {-0.673980117f, -0.738749504f},
{-0.678800762f, -0.734322488f}, {-0.683592319f, -0.729864061f},
{-0.688354552f, -0.725374401f}, {-0.693087339f, -0.720853567f},
{-0.697790444f, -0.716301918f}, {-0.702463686f, -0.711719632f},
{-0.707106769f, -0.707106769f}, {-0.711719632f, -0.702463686f},
{-0.716301918f, -0.697790444f}, {-0.720853567f, -0.693087339f},
{-0.725374401f, -0.688354552f}, {-0.729864061f, -0.683592319f},
{-0.734322488f, -0.678800762f}, {-0.738749504f, -0.673980117f},
{-0.743144810f, -0.669130623f}, {-0.747508347f, -0.664252460f},
{-0.751839817f, -0.659345806f}, {-0.756139100f, -0.654410958f},
{-0.760405958f, -0.649448037f}, {-0.764640272f, -0.644457340f},
{-0.768841803f, -0.639438987f}, {-0.773010433f, -0.634393275f},
{-0.777145982f, -0.629320383f}, {-0.781248152f, -0.624220550f},
{-0.785316944f, -0.619093955f}, {-0.789352059f, -0.613940835f},
{-0.793353319f, -0.608761430f}, {-0.797320664f, -0.603555918f},
{-0.801253796f, -0.598324597f}, {-0.805152655f, -0.593067646f},
{-0.809017003f, -0.587785244f}, {-0.812846661f, -0.582477689f},
{-0.816641569f, -0.577145219f}, {-0.820401430f, -0.571787953f},
{-0.824126184f, -0.566406250f}, {-0.827815652f, -0.561000228f},
{-0.831469595f, -0.555570245f}, {-0.835087955f, -0.550116420f},
{-0.838670552f, -0.544639051f}, {-0.842217207f, -0.539138317f},
{-0.845727801f, -0.533614516f}, {-0.849202156f, -0.528067827f},
{-0.852640152f, -0.522498548f}, {-0.856041610f, -0.516906917f},
{-0.859406412f, -0.511293113f}, {-0.862734377f, -0.505657375f},
{-0.866025388f, -0.500000000f}, {-0.869279325f, -0.494321197f},
{-0.872496009f, -0.488621235f}, {-0.875675321f, -0.482900351f},
{-0.878817141f, -0.477158755f}, {-0.881921291f, -0.471396744f},
{-0.884987652f, -0.465614527f}, {-0.888016105f, -0.459812373f},
{-0.891006529f, -0.453990489f}, {-0.893958807f, -0.448149204f},
{-0.896872759f, -0.442288697f}, {-0.899748266f, -0.436409235f},
{-0.902585268f, -0.430511087f}, {-0.905383646f, -0.424594522f},
{-0.908143163f, -0.418659747f}, {-0.910863817f, -0.412707031f},
{-0.913545430f, -0.406736642f}, {-0.916187942f, -0.400748819f},
{-0.918791234f, -0.394743860f}, {-0.921355128f, -0.388721973f},
{-0.923879504f, -0.382683426f}, {-0.926364362f, -0.376628488f},
{-0.928809524f, -0.370557427f}, {-0.931214929f, -0.364470512f},
{-0.933580399f, -0.358367950f}, {-0.935905933f, -0.352250040f},
{-0.938191354f, -0.346117049f}, {-0.940436542f, -0.339969248f},
{-0.942641497f, -0.333806872f}, {-0.944806039f, -0.327630192f},
{-0.946930110f, -0.321439475f}, {-0.949013650f, -0.315234989f},
{-0.951056540f, -0.309017003f}, {-0.953058660f, -0.302785784f},
{-0.955019951f, -0.296541572f}, {-0.956940353f, -0.290284663f},
{-0.958819747f, -0.284015357f}, {-0.960658073f, -0.277733833f},
{-0.962455213f, -0.271440446f}, {-0.964211166f, -0.265135437f},
{-0.965925813f, -0.258819044f}, {-0.967599094f, -0.252491564f},
{-0.969230890f, -0.246153295f}, {-0.970821202f, -0.239804462f},
{-0.972369909f, -0.233445361f}, {-0.973876953f, -0.227076262f},
{-0.975342333f, -0.220697433f}, {-0.976765871f, -0.214309156f},
{-0.978147626f, -0.207911685f}, {-0.979487419f, -0.201505318f},
{-0.980785251f, -0.195090324f}, {-0.982041121f, -0.188666970f},
{-0.983254910f, -0.182235524f}, {-0.984426558f, -0.175796285f},
{-0.985556066f, -0.169349506f}, {-0.986643314f, -0.162895471f},
{-0.987688363f, -0.156434461f}, {-0.988691032f, -0.149966761f},
{-0.989651382f, -0.143492624f}, {-0.990569353f, -0.137012348f},
{-0.991444886f, -0.130526185f}, {-0.992277920f, -0.124034449f},
{-0.993068457f, -0.117537394f}, {-0.993816435f, -0.111035310f},
{-0.994521916f, -0.104528464f}, {-0.995184720f, -0.0980171412f},
{-0.995804906f, -0.0915016159f}, {-0.996382475f, -0.0849821791f},
{-0.996917307f, -0.0784590989f}, {-0.997409463f, -0.0719326511f},
{-0.997858942f, -0.0654031262f}, {-0.998265624f, -0.0588708036f},
{-0.998629510f, -0.0523359552f}, {-0.998950660f, -0.0457988679f},
{-0.999229014f, -0.0392598175f}, {-0.999464571f, -0.0327190831f},
{-0.999657333f, -0.0261769481f}, {-0.999807239f, -0.0196336918f},
{-0.999914348f, -0.0130895954f}, {-0.999978602f, -0.00654493785f},
{-1.00000000f, -1.22464685e-16f}, {-0.999978602f, 0.00654493785f},
{-0.999914348f, 0.0130895954f}, {-0.999807239f, 0.0196336918f},
{-0.999657333f, 0.0261769481f}, {-0.999464571f, 0.0327190831f},
{-0.999229014f, 0.0392598175f}, {-0.998950660f, 0.0457988679f},
{-0.998629510f, 0.0523359552f}, {-0.998265624f, 0.0588708036f},
{-0.997858942f, 0.0654031262f}, {-0.997409463f, 0.0719326511f},
{-0.996917307f, 0.0784590989f}, {-0.996382475f, 0.0849821791f},
{-0.995804906f, 0.0915016159f}, {-0.995184720f, 0.0980171412f},
{-0.994521916f, 0.104528464f}, {-0.993816435f, 0.111035310f},
{-0.993068457f, 0.117537394f}, {-0.992277920f, 0.124034449f},
{-0.991444886f, 0.130526185f}, {-0.990569353f, 0.137012348f},
{-0.989651382f, 0.143492624f}, {-0.988691032f, 0.149966761f},
{-0.987688363f, 0.156434461f}, {-0.986643314f, 0.162895471f},
{-0.985556066f, 0.169349506f}, {-0.984426558f, 0.175796285f},
{-0.983254910f, 0.182235524f}, {-0.982041121f, 0.188666970f},
{-0.980785251f, 0.195090324f}, {-0.979487419f, 0.201505318f},
{-0.978147626f, 0.207911685f}, {-0.976765871f, 0.214309156f},
{-0.975342333f, 0.220697433f}, {-0.973876953f, 0.227076262f},
{-0.972369909f, 0.233445361f}, {-0.970821202f, 0.239804462f},
{-0.969230890f, 0.246153295f}, {-0.967599094f, 0.252491564f},
{-0.965925813f, 0.258819044f}, {-0.964211166f, 0.265135437f},
{-0.962455213f, 0.271440446f}, {-0.960658073f, 0.277733833f},
{-0.958819747f, 0.284015357f}, {-0.956940353f, 0.290284663f},
{-0.955019951f, 0.296541572f}, {-0.953058660f, 0.302785784f},
{-0.951056540f, 0.309017003f}, {-0.949013650f, 0.315234989f},
{-0.946930110f, 0.321439475f}, {-0.944806039f, 0.327630192f},
{-0.942641497f, 0.333806872f}, {-0.940436542f, 0.339969248f},
{-0.938191354f, 0.346117049f}, {-0.935905933f, 0.352250040f},
{-0.933580399f, 0.358367950f}, {-0.931214929f, 0.364470512f},
{-0.928809524f, 0.370557427f}, {-0.926364362f, 0.376628488f},
{-0.923879504f, 0.382683426f}, {-0.921355128f, 0.388721973f},
{-0.918791234f, 0.394743860f}, {-0.916187942f, 0.400748819f},
{-0.913545430f, 0.406736642f}, {-0.910863817f, 0.412707031f},
{-0.908143163f, 0.418659747f}, {-0.905383646f, 0.424594522f},
{-0.902585268f, 0.430511087f}, {-0.899748266f, 0.436409235f},
{-0.896872759f, 0.442288697f}, {-0.893958807f, 0.448149204f},
{-0.891006529f, 0.453990489f}, {-0.888016105f, 0.459812373f},
{-0.884987652f, 0.465614527f}, {-0.881921291f, 0.471396744f},
{-0.878817141f, 0.477158755f}, {-0.875675321f, 0.482900351f},
{-0.872496009f, 0.488621235f}, {-0.869279325f, 0.494321197f},
{-0.866025388f, 0.500000000f}, {-0.862734377f, 0.505657375f},
{-0.859406412f, 0.511293113f}, {-0.856041610f, 0.516906917f},
{-0.852640152f, 0.522498548f}, {-0.849202156f, 0.528067827f},
{-0.845727801f, 0.533614516f}, {-0.842217207f, 0.539138317f},
{-0.838670552f, 0.544639051f}, {-0.835087955f, 0.550116420f},
{-0.831469595f, 0.555570245f}, {-0.827815652f, 0.561000228f},
{-0.824126184f, 0.566406250f}, {-0.820401430f, 0.571787953f},
{-0.816641569f, 0.577145219f}, {-0.812846661f, 0.582477689f},
{-0.809017003f, 0.587785244f}, {-0.805152655f, 0.593067646f},
{-0.801253796f, 0.598324597f}, {-0.797320664f, 0.603555918f},
{-0.793353319f, 0.608761430f}, {-0.789352059f, 0.613940835f},
{-0.785316944f, 0.619093955f}, {-0.781248152f, 0.624220550f},
{-0.777145982f, 0.629320383f}, {-0.773010433f, 0.634393275f},
{-0.768841803f, 0.639438987f}, {-0.764640272f, 0.644457340f},
{-0.760405958f, 0.649448037f}, {-0.756139100f, 0.654410958f},
{-0.751839817f, 0.659345806f}, {-0.747508347f, 0.664252460f},
{-0.743144810f, 0.669130623f}, {-0.738749504f, 0.673980117f},
{-0.734322488f, 0.678800762f}, {-0.729864061f, 0.683592319f},
{-0.725374401f, 0.688354552f}, {-0.720853567f, 0.693087339f},
{-0.716301918f, 0.697790444f}, {-0.711719632f, 0.702463686f},
{-0.707106769f, 0.707106769f}, {-0.702463686f, 0.711719632f},
{-0.697790444f, 0.716301918f}, {-0.693087339f, 0.720853567f},
{-0.688354552f, 0.725374401f}, {-0.683592319f, 0.729864061f},
{-0.678800762f, 0.734322488f}, {-0.673980117f, 0.738749504f},
{-0.669130623f, 0.743144810f}, {-0.664252460f, 0.747508347f},
{-0.659345806f, 0.751839817f}, {-0.654410958f, 0.756139100f},
{-0.649448037f, 0.760405958f}, {-0.644457340f, 0.764640272f},
{-0.639438987f, 0.768841803f}, {-0.634393275f, 0.773010433f},
{-0.629320383f, 0.777145982f}, {-0.624220550f, 0.781248152f},
{-0.619093955f, 0.785316944f}, {-0.613940835f, 0.789352059f},
{-0.608761430f, 0.793353319f}, {-0.603555918f, 0.797320664f},
{-0.598324597f, 0.801253796f}, {-0.593067646f, 0.805152655f},
{-0.587785244f, 0.809017003f}, {-0.582477689f, 0.812846661f},
{-0.577145219f, 0.816641569f}, {-0.571787953f, 0.820401430f},
{-0.566406250f, 0.824126184f}, {-0.561000228f, 0.827815652f},
{-0.555570245f, 0.831469595f}, {-0.550116420f, 0.835087955f},
{-0.544639051f, 0.838670552f}, {-0.539138317f, 0.842217207f},
{-0.533614516f, 0.845727801f}, {-0.528067827f, 0.849202156f},
{-0.522498548f, 0.852640152f}, {-0.516906917f, 0.856041610f},
{-0.511293113f, 0.859406412f}, {-0.505657375f, 0.862734377f},
{-0.500000000f, 0.866025388f}, {-0.494321197f, 0.869279325f},
{-0.488621235f, 0.872496009f}, {-0.482900351f, 0.875675321f},
{-0.477158755f, 0.878817141f}, {-0.471396744f, 0.881921291f},
{-0.465614527f, 0.884987652f}, {-0.459812373f, 0.888016105f},
{-0.453990489f, 0.891006529f}, {-0.448149204f, 0.893958807f},
{-0.442288697f, 0.896872759f}, {-0.436409235f, 0.899748266f},
{-0.430511087f, 0.902585268f}, {-0.424594522f, 0.905383646f},
{-0.418659747f, 0.908143163f}, {-0.412707031f, 0.910863817f},
{-0.406736642f, 0.913545430f}, {-0.400748819f, 0.916187942f},
{-0.394743860f, 0.918791234f}, {-0.388721973f, 0.921355128f},
{-0.382683426f, 0.923879504f}, {-0.376628488f, 0.926364362f},
{-0.370557427f, 0.928809524f}, {-0.364470512f, 0.931214929f},
{-0.358367950f, 0.933580399f}, {-0.352250040f, 0.935905933f},
{-0.346117049f, 0.938191354f}, {-0.339969248f, 0.940436542f},
{-0.333806872f, 0.942641497f}, {-0.327630192f, 0.944806039f},
{-0.321439475f, 0.946930110f}, {-0.315234989f, 0.949013650f},
{-0.309017003f, 0.951056540f}, {-0.302785784f, 0.953058660f},
{-0.296541572f, 0.955019951f}, {-0.290284663f, 0.956940353f},
{-0.284015357f, 0.958819747f}, {-0.277733833f, 0.960658073f},
{-0.271440446f, 0.962455213f}, {-0.265135437f, 0.964211166f},
{-0.258819044f, 0.965925813f}, {-0.252491564f, 0.967599094f},
{-0.246153295f, 0.969230890f}, {-0.239804462f, 0.970821202f},
{-0.233445361f, 0.972369909f}, {-0.227076262f, 0.973876953f},
{-0.220697433f, 0.975342333f}, {-0.214309156f, 0.976765871f},
{-0.207911685f, 0.978147626f}, {-0.201505318f, 0.979487419f},
{-0.195090324f, 0.980785251f}, {-0.188666970f, 0.982041121f},
{-0.182235524f, 0.983254910f}, {-0.175796285f, 0.984426558f},
{-0.169349506f, 0.985556066f}, {-0.162895471f, 0.986643314f},
{-0.156434461f, 0.987688363f}, {-0.149966761f, 0.988691032f},
{-0.143492624f, 0.989651382f}, {-0.137012348f, 0.990569353f},
{-0.130526185f, 0.991444886f}, {-0.124034449f, 0.992277920f},
{-0.117537394f, 0.993068457f}, {-0.111035310f, 0.993816435f},
{-0.104528464f, 0.994521916f}, {-0.0980171412f, 0.995184720f},
{-0.0915016159f, 0.995804906f}, {-0.0849821791f, 0.996382475f},
{-0.0784590989f, 0.996917307f}, {-0.0719326511f, 0.997409463f},
{-0.0654031262f, 0.997858942f}, {-0.0588708036f, 0.998265624f},
{-0.0523359552f, 0.998629510f}, {-0.0457988679f, 0.998950660f},
{-0.0392598175f, 0.999229014f}, {-0.0327190831f, 0.999464571f},
{-0.0261769481f, 0.999657333f}, {-0.0196336918f, 0.999807239f},
{-0.0130895954f, 0.999914348f}, {-0.00654493785f, 0.999978602f},
{-1.83697015e-16f, 1.00000000f}, {0.00654493785f, 0.999978602f},
{0.0130895954f, 0.999914348f}, {0.0196336918f, 0.999807239f},
{0.0261769481f, 0.999657333f}, {0.0327190831f, 0.999464571f},
{0.0392598175f, 0.999229014f}, {0.0457988679f, 0.998950660f},
{0.0523359552f, 0.998629510f}, {0.0588708036f, 0.998265624f},
{0.0654031262f, 0.997858942f}, {0.0719326511f, 0.997409463f},
{0.0784590989f, 0.996917307f}, {0.0849821791f, 0.996382475f},
{0.0915016159f, 0.995804906f}, {0.0980171412f, 0.995184720f},
{0.104528464f, 0.994521916f}, {0.111035310f, 0.993816435f},
{0.117537394f, 0.993068457f}, {0.124034449f, 0.992277920f},
{0.130526185f, 0.991444886f}, {0.137012348f, 0.990569353f},
{0.143492624f, 0.989651382f}, {0.149966761f, 0.988691032f},
{0.156434461f, 0.987688363f}, {0.162895471f, 0.986643314f},
{0.169349506f, 0.985556066f}, {0.175796285f, 0.984426558f},
{0.182235524f, 0.983254910f}, {0.188666970f, 0.982041121f},
{0.195090324f, 0.980785251f}, {0.201505318f, 0.979487419f},
{0.207911685f, 0.978147626f}, {0.214309156f, 0.976765871f},
{0.220697433f, 0.975342333f}, {0.227076262f, 0.973876953f},
{0.233445361f, 0.972369909f}, {0.239804462f, 0.970821202f},
{0.246153295f, 0.969230890f}, {0.252491564f, 0.967599094f},
{0.258819044f, 0.965925813f}, {0.265135437f, 0.964211166f},
{0.271440446f, 0.962455213f}, {0.277733833f, 0.960658073f},
{0.284015357f, 0.958819747f}, {0.290284663f, 0.956940353f},
{0.296541572f, 0.955019951f}, {0.302785784f, 0.953058660f},
{0.309017003f, 0.951056540f}, {0.315234989f, 0.949013650f},
{0.321439475f, 0.946930110f}, {0.327630192f, 0.944806039f},
{0.333806872f, 0.942641497f}, {0.339969248f, 0.940436542f},
{0.346117049f, 0.938191354f}, {0.352250040f, 0.935905933f},
{0.358367950f, 0.933580399f}, {0.364470512f, 0.931214929f},
{0.370557427f, 0.928809524f}, {0.376628488f, 0.926364362f},
{0.382683426f, 0.923879504f}, {0.388721973f, 0.921355128f},
{0.394743860f, 0.918791234f}, {0.400748819f, 0.916187942f},
{0.406736642f, 0.913545430f}, {0.412707031f, 0.910863817f},
{0.418659747f, 0.908143163f}, {0.424594522f, 0.905383646f},
{0.430511087f, 0.902585268f}, {0.436409235f, 0.899748266f},
{0.442288697f, 0.896872759f}, {0.448149204f, 0.893958807f},
{0.453990489f, 0.891006529f}, {0.459812373f, 0.888016105f},
{0.465614527f, 0.884987652f}, {0.471396744f, 0.881921291f},
{0.477158755f, 0.878817141f}, {0.482900351f, 0.875675321f},
{0.488621235f, 0.872496009f}, {0.494321197f, 0.869279325f},
{0.500000000f, 0.866025388f}, {0.505657375f, 0.862734377f},
{0.511293113f, 0.859406412f}, {0.516906917f, 0.856041610f},
{0.522498548f, 0.852640152f}, {0.528067827f, 0.849202156f},
{0.533614516f, 0.845727801f}, {0.539138317f, 0.842217207f},
{0.544639051f, 0.838670552f}, {0.550116420f, 0.835087955f},
{0.555570245f, 0.831469595f}, {0.561000228f, 0.827815652f},
{0.566406250f, 0.824126184f}, {0.571787953f, 0.820401430f},
{0.577145219f, 0.816641569f}, {0.582477689f, 0.812846661f},
{0.587785244f, 0.809017003f}, {0.593067646f, 0.805152655f},
{0.598324597f, 0.801253796f}, {0.603555918f, 0.797320664f},
{0.608761430f, 0.793353319f}, {0.613940835f, 0.789352059f},
{0.619093955f, 0.785316944f}, {0.624220550f, 0.781248152f},
{0.629320383f, 0.777145982f}, {0.634393275f, 0.773010433f},
{0.639438987f, 0.768841803f}, {0.644457340f, 0.764640272f},
{0.649448037f, 0.760405958f}, {0.654410958f, 0.756139100f},
{0.659345806f, 0.751839817f}, {0.664252460f, 0.747508347f},
{0.669130623f, 0.743144810f}, {0.673980117f, 0.738749504f},
{0.678800762f, 0.734322488f}, {0.683592319f, 0.729864061f},
{0.688354552f, 0.725374401f}, {0.693087339f, 0.720853567f},
{0.697790444f, 0.716301918f}, {0.702463686f, 0.711719632f},
{0.707106769f, 0.707106769f}, {0.711719632f, 0.702463686f},
{0.716301918f, 0.697790444f}, {0.720853567f, 0.693087339f},
{0.725374401f, 0.688354552f}, {0.729864061f, 0.683592319f},
{0.734322488f, 0.678800762f}, {0.738749504f, 0.673980117f},
{0.743144810f, 0.669130623f}, {0.747508347f, 0.664252460f},
{0.751839817f, 0.659345806f}, {0.756139100f, 0.654410958f},
{0.760405958f, 0.649448037f}, {0.764640272f, 0.644457340f},
{0.768841803f, 0.639438987f}, {0.773010433f, 0.634393275f},
{0.777145982f, 0.629320383f}, {0.781248152f, 0.624220550f},
{0.785316944f, 0.619093955f}, {0.789352059f, 0.613940835f},
{0.793353319f, 0.608761430f}, {0.797320664f, 0.603555918f},
{0.801253796f, 0.598324597f}, {0.805152655f, 0.593067646f},
{0.809017003f, 0.587785244f}, {0.812846661f, 0.582477689f},
{0.816641569f, 0.577145219f}, {0.820401430f, 0.571787953f},
{0.824126184f, 0.566406250f}, {0.827815652f, 0.561000228f},
{0.831469595f, 0.555570245f}, {0.835087955f, 0.550116420f},
{0.838670552f, 0.544639051f}, {0.842217207f, 0.539138317f},
{0.845727801f, 0.533614516f}, {0.849202156f, 0.528067827f},
{0.852640152f, 0.522498548f}, {0.856041610f, 0.516906917f},
{0.859406412f, 0.511293113f}, {0.862734377f, 0.505657375f},
{0.866025388f, 0.500000000f}, {0.869279325f, 0.494321197f},
{0.872496009f, 0.488621235f}, {0.875675321f, 0.482900351f},
{0.878817141f, 0.477158755f}, {0.881921291f, 0.471396744f},
{0.884987652f, 0.465614527f}, {0.888016105f, 0.459812373f},
{0.891006529f, 0.453990489f}, {0.893958807f, 0.448149204f},
{0.896872759f, 0.442288697f}, {0.899748266f, 0.436409235f},
{0.902585268f, 0.430511087f}, {0.905383646f, 0.424594522f},
{0.908143163f, 0.418659747f}, {0.910863817f, 0.412707031f},
{0.913545430f, 0.406736642f}, {0.916187942f, 0.400748819f},
{0.918791234f, 0.394743860f}, {0.921355128f, 0.388721973f},
{0.923879504f, 0.382683426f}, {0.926364362f, 0.376628488f},
{0.928809524f, 0.370557427f}, {0.931214929f, 0.364470512f},
{0.933580399f, 0.358367950f}, {0.935905933f, 0.352250040f},
{0.938191354f, 0.346117049f}, {0.940436542f, 0.339969248f},
{0.942641497f, 0.333806872f}, {0.944806039f, 0.327630192f},
{0.946930110f, 0.321439475f}, {0.949013650f, 0.315234989f},
{0.951056540f, 0.309017003f}, {0.953058660f, 0.302785784f},
{0.955019951f, 0.296541572f}, {0.956940353f, 0.290284663f},
{0.958819747f, 0.284015357f}, {0.960658073f, 0.277733833f},
{0.962455213f, 0.271440446f}, {0.964211166f, 0.265135437f},
{0.965925813f, 0.258819044f}, {0.967599094f, 0.252491564f},
{0.969230890f, 0.246153295f}, {0.970821202f, 0.239804462f},
{0.972369909f, 0.233445361f}, {0.973876953f, 0.227076262f},
{0.975342333f, 0.220697433f}, {0.976765871f, 0.214309156f},
{0.978147626f, 0.207911685f}, {0.979487419f, 0.201505318f},
{0.980785251f, 0.195090324f}, {0.982041121f, 0.188666970f},
{0.983254910f, 0.182235524f}, {0.984426558f, 0.175796285f},
{0.985556066f, 0.169349506f}, {0.986643314f, 0.162895471f},
{0.987688363f, 0.156434461f}, {0.988691032f, 0.149966761f},
{0.989651382f, 0.143492624f}, {0.990569353f, 0.137012348f},
{0.991444886f, 0.130526185f}, {0.992277920f, 0.124034449f},
{0.993068457f, 0.117537394f}, {0.993816435f, 0.111035310f},
{0.994521916f, 0.104528464f}, {0.995184720f, 0.0980171412f},
{0.995804906f, 0.0915016159f}, {0.996382475f, 0.0849821791f},
{0.996917307f, 0.0784590989f}, {0.997409463f, 0.0719326511f},
{0.997858942f, 0.0654031262f}, {0.998265624f, 0.0588708036f},
{0.998629510f, 0.0523359552f}, {0.998950660f, 0.0457988679f},
{0.999229014f, 0.0392598175f}, {0.999464571f, 0.0327190831f},
{0.999657333f, 0.0261769481f}, {0.999807239f, 0.0196336918f},
{0.999914348f, 0.0130895954f}, {0.999978602f, 0.00654493785f},
};
const kiss_fft_state rnn_kfft = {
960, /* nfft */
0.0010416667f, /* scale */
-1, /* shift */
{5, 192, 3, 64, 4, 16, 4, 4, 4, 1, 0, 0, 0, 0, 0, 0, }, /* factors */
fft_bitrev, /* bitrev*/
fft_twiddles, /* twiddles*/
(arch_fft_state *)&arch_fft, /* arch_fft*/
};
const float rnn_half_window[] = {
4.20549168e-06f, 3.78491532e-05f, 0.000105135041f, 0.000206060256f, 0.000340620492f,
0.000508809986f, 0.000710621476f, 0.000946046319f, 0.00121507444f, 0.00151769421f,
0.00185389258f, 0.00222365512f, 0.00262696599f, 0.00306380726f, 0.00353416055f,
0.00403800514f, 0.00457531959f, 0.00514607970f, 0.00575026125f, 0.00638783723f,
0.00705878017f, 0.00776306028f, 0.00850064680f, 0.00927150715f, 0.0100756064f,
0.0109129101f, 0.0117833801f, 0.0126869772f, 0.0136236614f, 0.0145933898f,
0.0155961197f, 0.0166318044f, 0.0177003983f, 0.0188018531f, 0.0199361145f,
0.0211031344f, 0.0223028567f, 0.0235352255f, 0.0248001851f, 0.0260976739f,
0.0274276342f, 0.0287899990f, 0.0301847085f, 0.0316116922f, 0.0330708846f,
0.0345622115f, 0.0360856056f, 0.0376409888f, 0.0392282903f, 0.0408474281f,
0.0424983241f, 0.0441808924f, 0.0458950549f, 0.0476407260f, 0.0494178124f,
0.0512262285f, 0.0530658774f, 0.0549366735f, 0.0568385124f, 0.0587713011f,
0.0607349351f, 0.0627293140f, 0.0647543296f, 0.0668098852f, 0.0688958541f,
0.0710121393f, 0.0731586292f, 0.0753351897f, 0.0775417164f, 0.0797780901f,
0.0820441842f, 0.0843398646f, 0.0866650119f, 0.0890194997f, 0.0914031938f,
0.0938159525f, 0.0962576419f, 0.0987281203f, 0.101227246f, 0.103754878f,
0.106310867f, 0.108895063f, 0.111507311f, 0.114147455f, 0.116815343f,
0.119510807f, 0.122233689f, 0.124983832f, 0.127761051f, 0.130565181f,
0.133396059f, 0.136253506f, 0.139137328f, 0.142047361f, 0.144983411f,
0.147945285f, 0.150932819f, 0.153945804f, 0.156984031f, 0.160047337f,
0.163135484f, 0.166248307f, 0.169385567f, 0.172547072f, 0.175732598f,
0.178941950f, 0.182174906f, 0.185431242f, 0.188710734f, 0.192013159f,
0.195338294f, 0.198685899f, 0.202055752f, 0.205447599f, 0.208861232f,
0.212296382f, 0.215752810f, 0.219230279f, 0.222728521f, 0.226247311f,
0.229786381f, 0.233345464f, 0.236924306f, 0.240522653f, 0.244140238f,
0.247776777f, 0.251432031f, 0.255105674f, 0.258797467f, 0.262507141f,
0.266234398f, 0.269978970f, 0.273740560f, 0.277518868f, 0.281313598f,
0.285124481f, 0.288951218f, 0.292793512f, 0.296651065f, 0.300523549f,
0.304410696f, 0.308312178f, 0.312227666f, 0.316156894f, 0.320099503f,
0.324055225f, 0.328023702f, 0.332004637f, 0.335997701f, 0.340002567f,
0.344018906f, 0.348046392f, 0.352084726f, 0.356133521f, 0.360192508f,
0.364261299f, 0.368339598f, 0.372427016f, 0.376523286f, 0.380627990f,
0.384740859f, 0.388861477f, 0.392989576f, 0.397124738f, 0.401266664f,
0.405414969f, 0.409569323f, 0.413729399f, 0.417894781f, 0.422065198f,
0.426240236f, 0.430419534f, 0.434602767f, 0.438789606f, 0.442979604f,
0.447172493f, 0.451367885f, 0.455565393f, 0.459764689f, 0.463965416f,
0.468167186f, 0.472369671f, 0.476572484f, 0.480775267f, 0.484977663f,
0.489179343f, 0.493379891f, 0.497579008f, 0.501776278f, 0.505971372f,
0.510163903f, 0.514353573f, 0.518539906f, 0.522722721f, 0.526901484f,
0.531075954f, 0.535245717f, 0.539410412f, 0.543569744f, 0.547723293f,
0.551870763f, 0.556011736f, 0.560145974f, 0.564273000f, 0.568392515f,
0.572504222f, 0.576607704f, 0.580702662f, 0.584788740f, 0.588865638f,
0.592932940f, 0.596990347f, 0.601037502f, 0.605074167f, 0.609099925f,
0.613114417f, 0.617117405f, 0.621108532f, 0.625087440f, 0.629053831f,
0.633007407f, 0.636947870f, 0.640874863f, 0.644788086f, 0.648687243f,
0.652572036f, 0.656442165f, 0.660297334f, 0.664137185f, 0.667961538f,
0.671769977f, 0.675562322f, 0.679338276f, 0.683097482f, 0.686839759f,
0.690564752f, 0.694272280f, 0.697961986f, 0.701633692f, 0.705287039f,
0.708921850f, 0.712537885f, 0.716134787f, 0.719712436f, 0.723270535f,
0.726808906f, 0.730327189f, 0.733825266f, 0.737302899f, 0.740759790f,
0.744195819f, 0.747610688f, 0.751004279f, 0.754376352f, 0.757726669f,
0.761055112f, 0.764361382f, 0.767645359f, 0.770906866f, 0.774145722f,
0.777361751f, 0.780554771f, 0.783724606f, 0.786871076f, 0.789994121f,
0.793093503f, 0.796169102f, 0.799220800f, 0.802248418f, 0.805251837f,
0.808230937f, 0.811185598f, 0.814115703f, 0.817021132f, 0.819901764f,
0.822757542f, 0.825588286f, 0.828393936f, 0.831174433f, 0.833929658f,
0.836659551f, 0.839363992f, 0.842042983f, 0.844696403f, 0.847324252f,
0.849926353f, 0.852502763f, 0.855053425f, 0.857578218f, 0.860077202f,
0.862550259f, 0.864997447f, 0.867418647f, 0.869813919f, 0.872183204f,
0.874526560f, 0.876843870f, 0.879135191f, 0.881400526f, 0.883639932f,
0.885853291f, 0.888040781f, 0.890202343f, 0.892337978f, 0.894447744f,
0.896531701f, 0.898589849f, 0.900622249f, 0.902628958f, 0.904610038f,
0.906565487f, 0.908495426f, 0.910399914f, 0.912279010f, 0.914132774f,
0.915961266f, 0.917764664f, 0.919542909f, 0.921296239f, 0.923024654f,
0.924728215f, 0.926407158f, 0.928061485f, 0.929691315f, 0.931296766f,
0.932878017f, 0.934435070f, 0.935968161f, 0.937477291f, 0.938962698f,
0.940424502f, 0.941862822f, 0.943277776f, 0.944669485f, 0.946038187f,
0.947383940f, 0.948706925f, 0.950007319f, 0.951285243f, 0.952540874f,
0.953774393f, 0.954985917f, 0.956175685f, 0.957343817f, 0.958490491f,
0.959615886f, 0.960720181f, 0.961803555f, 0.962866247f, 0.963908315f,
0.964930058f, 0.965931594f, 0.966913164f, 0.967874944f, 0.968817174f,
0.969739914f, 0.970643520f, 0.971528113f, 0.972393870f, 0.973241091f,
0.974069893f, 0.974880517f, 0.975673139f, 0.976447999f, 0.977205336f,
0.977945268f, 0.978668094f, 0.979374051f, 0.980063200f, 0.980735898f,
0.981392324f, 0.982032716f, 0.982657254f, 0.983266115f, 0.983859658f,
0.984437943f, 0.985001266f, 0.985549867f, 0.986083925f, 0.986603677f,
0.987109363f, 0.987601161f, 0.988079309f, 0.988544047f, 0.988995552f,
0.989434063f, 0.989859879f, 0.990273118f, 0.990674019f, 0.991062820f,
0.991439700f, 0.991804957f, 0.992158771f, 0.992501318f, 0.992832899f,
0.993153632f, 0.993463814f, 0.993763626f, 0.994053245f, 0.994332969f,
0.994602919f, 0.994863331f, 0.995114446f, 0.995356441f, 0.995589554f,
0.995813966f, 0.996029854f, 0.996237516f, 0.996437073f, 0.996628702f,
0.996812642f, 0.996989131f, 0.997158289f, 0.997320294f, 0.997475445f,
0.997623861f, 0.997765720f, 0.997901261f, 0.998030603f, 0.998153925f,
0.998271465f, 0.998383403f, 0.998489857f, 0.998591006f, 0.998687088f,
0.998778164f, 0.998864532f, 0.998946249f, 0.999023557f, 0.999096513f,
0.999165416f, 0.999230266f, 0.999291301f, 0.999348700f, 0.999402523f,
0.999453008f, 0.999500215f, 0.999544322f, 0.999585509f, 0.999623775f,
0.999659419f, 0.999692440f, 0.999723017f, 0.999751270f, 0.999777317f,
0.999801278f, 0.999823213f, 0.999843359f, 0.999861658f, 0.999878347f,
0.999893486f, 0.999907196f, 0.999919534f, 0.999930561f, 0.999940455f,
0.999949217f, 0.999957025f, 0.999963880f, 0.999969840f, 0.999975085f,
0.999979615f, 0.999983490f, 0.999986768f, 0.999989510f, 0.999991834f,
0.999993742f, 0.999995291f, 0.999996543f, 0.999997556f, 0.999998271f,
0.999998868f, 0.999999285f, 0.999999523f, 0.999999762f, 0.999999881f,
0.999999940f, 1.00000000f, 1.00000000f, 1.00000000f, 1.00000000f,
};
const float rnn_dct_table[] = {
0.707106769f, 0.998795450f, 0.995184720f, 0.989176512f, 0.980785251f,
0.970031261f, 0.956940353f, 0.941544056f, 0.923879504f, 0.903989315f,
0.881921291f, 0.857728601f, 0.831469595f, 0.803207517f, 0.773010433f,
0.740951121f, 0.707106769f, 0.671558976f, 0.634393275f, 0.595699310f,
0.555570245f, 0.514102757f, 0.471396744f, 0.427555084f, 0.382683426f,
0.336889863f, 0.290284663f, 0.242980182f, 0.195090324f, 0.146730468f,
0.0980171412f, 0.0490676761f, 0.707106769f, 0.989176512f, 0.956940353f,
0.903989315f, 0.831469595f, 0.740951121f, 0.634393275f, 0.514102757f,
0.382683426f, 0.242980182f, 0.0980171412f, -0.0490676761f, -0.195090324f,
-0.336889863f, -0.471396744f, -0.595699310f, -0.707106769f, -0.803207517f,
-0.881921291f, -0.941544056f, -0.980785251f, -0.998795450f, -0.995184720f,
-0.970031261f, -0.923879504f, -0.857728601f, -0.773010433f, -0.671558976f,
-0.555570245f, -0.427555084f, -0.290284663f, -0.146730468f, 0.707106769f,
0.970031261f, 0.881921291f, 0.740951121f, 0.555570245f, 0.336889863f,
0.0980171412f, -0.146730468f, -0.382683426f, -0.595699310f, -0.773010433f,
-0.903989315f, -0.980785251f, -0.998795450f, -0.956940353f, -0.857728601f,
-0.707106769f, -0.514102757f, -0.290284663f, -0.0490676761f, 0.195090324f,
0.427555084f, 0.634393275f, 0.803207517f, 0.923879504f, 0.989176512f,
0.995184720f, 0.941544056f, 0.831469595f, 0.671558976f, 0.471396744f,
0.242980182f, 0.707106769f, 0.941544056f, 0.773010433f, 0.514102757f,
0.195090324f, -0.146730468f, -0.471396744f, -0.740951121f, -0.923879504f,
-0.998795450f, -0.956940353f, -0.803207517f, -0.555570245f, -0.242980182f,
0.0980171412f, 0.427555084f, 0.707106769f, 0.903989315f, 0.995184720f,
0.970031261f, 0.831469595f, 0.595699310f, 0.290284663f, -0.0490676761f,
-0.382683426f, -0.671558976f, -0.881921291f, -0.989176512f, -0.980785251f,
-0.857728601f, -0.634393275f, -0.336889863f, 0.707106769f, 0.903989315f,
0.634393275f, 0.242980182f, -0.195090324f, -0.595699310f, -0.881921291f,
-0.998795450f, -0.923879504f, -0.671558976f, -0.290284663f, 0.146730468f,
0.555570245f, 0.857728601f, 0.995184720f, 0.941544056f, 0.707106769f,
0.336889863f, -0.0980171412f, -0.514102757f, -0.831469595f, -0.989176512f,
-0.956940353f, -0.740951121f, -0.382683426f, 0.0490676761f, 0.471396744f,
0.803207517f, 0.980785251f, 0.970031261f, 0.773010433f, 0.427555084f,
0.707106769f, 0.857728601f, 0.471396744f, -0.0490676761f, -0.555570245f,
-0.903989315f, -0.995184720f, -0.803207517f, -0.382683426f, 0.146730468f,
0.634393275f, 0.941544056f, 0.980785251f, 0.740951121f, 0.290284663f,
-0.242980182f, -0.707106769f, -0.970031261f, -0.956940353f, -0.671558976f,
-0.195090324f, 0.336889863f, 0.773010433f, 0.989176512f, 0.923879504f,
0.595699310f, 0.0980171412f, -0.427555084f, -0.831469595f, -0.998795450f,
-0.881921291f, -0.514102757f, 0.707106769f, 0.803207517f, 0.290284663f,
-0.336889863f, -0.831469595f, -0.998795450f, -0.773010433f, -0.242980182f,
0.382683426f, 0.857728601f, 0.995184720f, 0.740951121f, 0.195090324f,
-0.427555084f, -0.881921291f, -0.989176512f, -0.707106769f, -0.146730468f,
0.471396744f, 0.903989315f, 0.980785251f, 0.671558976f, 0.0980171412f,
-0.514102757f, -0.923879504f, -0.970031261f, -0.634393275f, -0.0490676761f,
0.555570245f, 0.941544056f, 0.956940353f, 0.595699310f, 0.707106769f,
0.740951121f, 0.0980171412f, -0.595699310f, -0.980785251f, -0.857728601f,
-0.290284663f, 0.427555084f, 0.923879504f, 0.941544056f, 0.471396744f,
-0.242980182f, -0.831469595f, -0.989176512f, -0.634393275f, 0.0490676761f,
0.707106769f, 0.998795450f, 0.773010433f, 0.146730468f, -0.555570245f,
-0.970031261f, -0.881921291f, -0.336889863f, 0.382683426f, 0.903989315f,
0.956940353f, 0.514102757f, -0.195090324f, -0.803207517f, -0.995184720f,
-0.671558976f, 0.707106769f, 0.671558976f, -0.0980171412f, -0.803207517f,
-0.980785251f, -0.514102757f, 0.290284663f, 0.903989315f, 0.923879504f,
0.336889863f, -0.471396744f, -0.970031261f, -0.831469595f, -0.146730468f,
0.634393275f, 0.998795450f, 0.707106769f, -0.0490676761f, -0.773010433f,
-0.989176512f, -0.555570245f, 0.242980182f, 0.881921291f, 0.941544056f,
0.382683426f, -0.427555084f, -0.956940353f, -0.857728601f, -0.195090324f,
0.595699310f, 0.995184720f, 0.740951121f, 0.707106769f, 0.595699310f,
-0.290284663f, -0.941544056f, -0.831469595f, -0.0490676761f, 0.773010433f,
0.970031261f, 0.382683426f, -0.514102757f, -0.995184720f, -0.671558976f,
0.195090324f, 0.903989315f, 0.881921291f, 0.146730468f, -0.707106769f,
-0.989176512f, -0.471396744f, 0.427555084f, 0.980785251f, 0.740951121f,
-0.0980171412f, -0.857728601f, -0.923879504f, -0.242980182f, 0.634393275f,
0.998795450f, 0.555570245f, -0.336889863f, -0.956940353f, -0.803207517f,
0.707106769f, 0.514102757f, -0.471396744f, -0.998795450f, -0.555570245f,
0.427555084f, 0.995184720f, 0.595699310f, -0.382683426f, -0.989176512f,
-0.634393275f, 0.336889863f, 0.980785251f, 0.671558976f, -0.290284663f,
-0.970031261f, -0.707106769f, 0.242980182f, 0.956940353f, 0.740951121f,
-0.195090324f, -0.941544056f, -0.773010433f, 0.146730468f, 0.923879504f,
0.803207517f, -0.0980171412f, -0.903989315f, -0.831469595f, 0.0490676761f,
0.881921291f, 0.857728601f, 0.707106769f, 0.427555084f, -0.634393275f,
-0.970031261f, -0.195090324f, 0.803207517f, 0.881921291f, -0.0490676761f,
-0.923879504f, -0.740951121f, 0.290284663f, 0.989176512f, 0.555570245f,
-0.514102757f, -0.995184720f, -0.336889863f, 0.707106769f, 0.941544056f,
0.0980171412f, -0.857728601f, -0.831469595f, 0.146730468f, 0.956940353f,
0.671558976f, -0.382683426f, -0.998795450f, -0.471396744f, 0.595699310f,
0.980785251f, 0.242980182f, -0.773010433f, -0.903989315f, 0.707106769f,
0.336889863f, -0.773010433f, -0.857728601f, 0.195090324f, 0.989176512f,
0.471396744f, -0.671558976f, -0.923879504f, 0.0490676761f, 0.956940353f,
0.595699310f, -0.555570245f, -0.970031261f, -0.0980171412f, 0.903989315f,
0.707106769f, -0.427555084f, -0.995184720f, -0.242980182f, 0.831469595f,
0.803207517f, -0.290284663f, -0.998795450f, -0.382683426f, 0.740951121f,
0.881921291f, -0.146730468f, -0.980785251f, -0.514102757f, 0.634393275f,
0.941544056f, 0.707106769f, 0.242980182f, -0.881921291f, -0.671558976f,
0.555570245f, 0.941544056f, -0.0980171412f, -0.989176512f, -0.382683426f,
0.803207517f, 0.773010433f, -0.427555084f, -0.980785251f, -0.0490676761f,
0.956940353f, 0.514102757f, -0.707106769f, -0.857728601f, 0.290284663f,
0.998795450f, 0.195090324f, -0.903989315f, -0.634393275f, 0.595699310f,
0.923879504f, -0.146730468f, -0.995184720f, -0.336889863f, 0.831469595f,
0.740951121f, -0.471396744f, -0.970031261f, 0.707106769f, 0.146730468f,
-0.956940353f, -0.427555084f, 0.831469595f, 0.671558976f, -0.634393275f,
-0.857728601f, 0.382683426f, 0.970031261f, -0.0980171412f, -0.998795450f,
-0.195090324f, 0.941544056f, 0.471396744f, -0.803207517f, -0.707106769f,
0.595699310f, 0.881921291f, -0.336889863f, -0.980785251f, 0.0490676761f,
0.995184720f, 0.242980182f, -0.923879504f, -0.514102757f, 0.773010433f,
0.740951121f, -0.555570245f, -0.903989315f, 0.290284663f, 0.989176512f,
0.707106769f, 0.0490676761f, -0.995184720f, -0.146730468f, 0.980785251f,
0.242980182f, -0.956940353f, -0.336889863f, 0.923879504f, 0.427555084f,
-0.881921291f, -0.514102757f, 0.831469595f, 0.595699310f, -0.773010433f,
-0.671558976f, 0.707106769f, 0.740951121f, -0.634393275f, -0.803207517f,
0.555570245f, 0.857728601f, -0.471396744f, -0.903989315f, 0.382683426f,
0.941544056f, -0.290284663f, -0.970031261f, 0.195090324f, 0.989176512f,
-0.0980171412f, -0.998795450f, 0.707106769f, -0.0490676761f, -0.995184720f,
0.146730468f, 0.980785251f, -0.242980182f, -0.956940353f, 0.336889863f,
0.923879504f, -0.427555084f, -0.881921291f, 0.514102757f, 0.831469595f,
-0.595699310f, -0.773010433f, 0.671558976f, 0.707106769f, -0.740951121f,
-0.634393275f, 0.803207517f, 0.555570245f, -0.857728601f, -0.471396744f,
0.903989315f, 0.382683426f, -0.941544056f, -0.290284663f, 0.970031261f,
0.195090324f, -0.989176512f, -0.0980171412f, 0.998795450f, 0.707106769f,
-0.146730468f, -0.956940353f, 0.427555084f, 0.831469595f, -0.671558976f,
-0.634393275f, 0.857728601f, 0.382683426f, -0.970031261f, -0.0980171412f,
0.998795450f, -0.195090324f, -0.941544056f, 0.471396744f, 0.803207517f,
-0.707106769f, -0.595699310f, 0.881921291f, 0.336889863f, -0.980785251f,
-0.0490676761f, 0.995184720f, -0.242980182f, -0.923879504f, 0.514102757f,
0.773010433f, -0.740951121f, -0.555570245f, 0.903989315f, 0.290284663f,
-0.989176512f, 0.707106769f, -0.242980182f, -0.881921291f, 0.671558976f,
0.555570245f, -0.941544056f, -0.0980171412f, 0.989176512f, -0.382683426f,
-0.803207517f, 0.773010433f, 0.427555084f, -0.980785251f, 0.0490676761f,
0.956940353f, -0.514102757f, -0.707106769f, 0.857728601f, 0.290284663f,
-0.998795450f, 0.195090324f, 0.903989315f, -0.634393275f, -0.595699310f,
0.923879504f, 0.146730468f, -0.995184720f, 0.336889863f, 0.831469595f,
-0.740951121f, -0.471396744f, 0.970031261f, 0.707106769f, -0.336889863f,
-0.773010433f, 0.857728601f, 0.195090324f, -0.989176512f, 0.471396744f,
0.671558976f, -0.923879504f, -0.0490676761f, 0.956940353f, -0.595699310f,
-0.555570245f, 0.970031261f, -0.0980171412f, -0.903989315f, 0.707106769f,
0.427555084f, -0.995184720f, 0.242980182f, 0.831469595f, -0.803207517f,
-0.290284663f, 0.998795450f, -0.382683426f, -0.740951121f, 0.881921291f,
0.146730468f, -0.980785251f, 0.514102757f, 0.634393275f, -0.941544056f,
0.707106769f, -0.427555084f, -0.634393275f, 0.970031261f, -0.195090324f,
-0.803207517f, 0.881921291f, 0.0490676761f, -0.923879504f, 0.740951121f,
0.290284663f, -0.989176512f, 0.555570245f, 0.514102757f, -0.995184720f,
0.336889863f, 0.707106769f, -0.941544056f, 0.0980171412f, 0.857728601f,
-0.831469595f, -0.146730468f, 0.956940353f, -0.671558976f, -0.382683426f,
0.998795450f, -0.471396744f, -0.595699310f, 0.980785251f, -0.242980182f,
-0.773010433f, 0.903989315f, 0.707106769f, -0.514102757f, -0.471396744f,
0.998795450f, -0.555570245f, -0.427555084f, 0.995184720f, -0.595699310f,
-0.382683426f, 0.989176512f, -0.634393275f, -0.336889863f, 0.980785251f,
-0.671558976f, -0.290284663f, 0.970031261f, -0.707106769f, -0.242980182f,
0.956940353f, -0.740951121f, -0.195090324f, 0.941544056f, -0.773010433f,
-0.146730468f, 0.923879504f, -0.803207517f, -0.0980171412f, 0.903989315f,
-0.831469595f, -0.0490676761f, 0.881921291f, -0.857728601f, 0.707106769f,
-0.595699310f, -0.290284663f, 0.941544056f, -0.831469595f, 0.0490676761f,
0.773010433f, -0.970031261f, 0.382683426f, 0.514102757f, -0.995184720f,
0.671558976f, 0.195090324f, -0.903989315f, 0.881921291f, -0.146730468f,
-0.707106769f, 0.989176512f, -0.471396744f, -0.427555084f, 0.980785251f,
-0.740951121f, -0.0980171412f, 0.857728601f, -0.923879504f, 0.242980182f,
0.634393275f, -0.998795450f, 0.555570245f, 0.336889863f, -0.956940353f,
0.803207517f, 0.707106769f, -0.671558976f, -0.0980171412f, 0.803207517f,
-0.980785251f, 0.514102757f, 0.290284663f, -0.903989315f, 0.923879504f,
-0.336889863f, -0.471396744f, 0.970031261f, -0.831469595f, 0.146730468f,
0.634393275f, -0.998795450f, 0.707106769f, 0.0490676761f, -0.773010433f,
0.989176512f, -0.555570245f, -0.242980182f, 0.881921291f, -0.941544056f,
0.382683426f, 0.427555084f, -0.956940353f, 0.857728601f, -0.195090324f,
-0.595699310f, 0.995184720f, -0.740951121f, 0.707106769f, -0.740951121f,
0.0980171412f, 0.595699310f, -0.980785251f, 0.857728601f, -0.290284663f,
-0.427555084f, 0.923879504f, -0.941544056f, 0.471396744f, 0.242980182f,
-0.831469595f, 0.989176512f, -0.634393275f, -0.0490676761f, 0.707106769f,
-0.998795450f, 0.773010433f, -0.146730468f, -0.555570245f, 0.970031261f,
-0.881921291f, 0.336889863f, 0.382683426f, -0.903989315f, 0.956940353f,
-0.514102757f, -0.195090324f, 0.803207517f, -0.995184720f, 0.671558976f,
0.707106769f, -0.803207517f, 0.290284663f, 0.336889863f, -0.831469595f,
0.998795450f, -0.773010433f, 0.242980182f, 0.382683426f, -0.857728601f,
0.995184720f, -0.740951121f, 0.195090324f, 0.427555084f, -0.881921291f,
0.989176512f, -0.707106769f, 0.146730468f, 0.471396744f, -0.903989315f,
0.980785251f, -0.671558976f, 0.0980171412f, 0.514102757f, -0.923879504f,
0.970031261f, -0.634393275f, 0.0490676761f, 0.555570245f, -0.941544056f,
0.956940353f, -0.595699310f, 0.707106769f, -0.857728601f, 0.471396744f,
0.0490676761f, -0.555570245f, 0.903989315f, -0.995184720f, 0.803207517f,
-0.382683426f, -0.146730468f, 0.634393275f, -0.941544056f, 0.980785251f,
-0.740951121f, 0.290284663f, 0.242980182f, -0.707106769f, 0.970031261f,
-0.956940353f, 0.671558976f, -0.195090324f, -0.336889863f, 0.773010433f,
-0.989176512f, 0.923879504f, -0.595699310f, 0.0980171412f, 0.427555084f,
-0.831469595f, 0.998795450f, -0.881921291f, 0.514102757f, 0.707106769f,
-0.903989315f, 0.634393275f, -0.242980182f, -0.195090324f, 0.595699310f,
-0.881921291f, 0.998795450f, -0.923879504f, 0.671558976f, -0.290284663f,
-0.146730468f, 0.555570245f, -0.857728601f, 0.995184720f, -0.941544056f,
0.707106769f, -0.336889863f, -0.0980171412f, 0.514102757f, -0.831469595f,
0.989176512f, -0.956940353f, 0.740951121f, -0.382683426f, -0.0490676761f,
0.471396744f, -0.803207517f, 0.980785251f, -0.970031261f, 0.773010433f,
-0.427555084f, 0.707106769f, -0.941544056f, 0.773010433f, -0.514102757f,
0.195090324f, 0.146730468f, -0.471396744f, 0.740951121f, -0.923879504f,
0.998795450f, -0.956940353f, 0.803207517f, -0.555570245f, 0.242980182f,
0.0980171412f, -0.427555084f, 0.707106769f, -0.903989315f, 0.995184720f,
-0.970031261f, 0.831469595f, -0.595699310f, 0.290284663f, 0.0490676761f,
-0.382683426f, 0.671558976f, -0.881921291f, 0.989176512f, -0.980785251f,
0.857728601f, -0.634393275f, 0.336889863f, 0.707106769f, -0.970031261f,
0.881921291f, -0.740951121f, 0.555570245f, -0.336889863f, 0.0980171412f,
0.146730468f, -0.382683426f, 0.595699310f, -0.773010433f, 0.903989315f,
-0.980785251f, 0.998795450f, -0.956940353f, 0.857728601f, -0.707106769f,
0.514102757f, -0.290284663f, 0.0490676761f, 0.195090324f, -0.427555084f,
0.634393275f, -0.803207517f, 0.923879504f, -0.989176512f, 0.995184720f,
-0.941544056f, 0.831469595f, -0.671558976f, 0.471396744f, -0.242980182f,
0.707106769f, -0.989176512f, 0.956940353f, -0.903989315f, 0.831469595f,
-0.740951121f, 0.634393275f, -0.514102757f, 0.382683426f, -0.242980182f,
0.0980171412f, 0.0490676761f, -0.195090324f, 0.336889863f, -0.471396744f,
0.595699310f, -0.707106769f, 0.803207517f, -0.881921291f, 0.941544056f,
-0.980785251f, 0.998795450f, -0.995184720f, 0.970031261f, -0.923879504f,
0.857728601f, -0.773010433f, 0.671558976f, -0.555570245f, 0.427555084f,
-0.290284663f, 0.146730468f, 0.707106769f, -0.998795450f, 0.995184720f,
-0.989176512f, 0.980785251f, -0.970031261f, 0.956940353f, -0.941544056f,
0.923879504f, -0.903989315f, 0.881921291f, -0.857728601f, 0.831469595f,
-0.803207517f, 0.773010433f, -0.740951121f, 0.707106769f, -0.671558976f,
0.634393275f, -0.595699310f, 0.555570245f, -0.514102757f, 0.471396744f,
-0.427555084f, 0.382683426f, -0.336889863f, 0.290284663f, -0.242980182f,
0.195090324f, -0.146730468f, 0.0980171412f, -0.0490676761f, };

388
third_party/rnnoise/src/vec.h vendored Normal file
View File

@@ -0,0 +1,388 @@
/* Copyright (c) 2018 Mozilla
2008-2011 Octasic Inc.
2012-2017 Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef VEC_H
#define VEC_H
#include "opus_types.h"
#include "common.h"
#include <math.h>
#include "arch.h"
#include "x86/x86_arch_macros.h"
#if defined(__AVX__) || defined(__SSE2__)
#include "vec_avx.h"
#elif (defined(__ARM_NEON__) || defined(__ARM_NEON)) && !defined(DISABLE_NEON)
#include "vec_neon.h"
#else
#define MAX_INPUTS (2048)
#define NO_OPTIMIZATIONS
static inline void sgemv16x1(float *out, const float *weights, int rows, int cols, int col_stride, const float *x)
{
int i, j;
RNN_CLEAR(out, rows);
for (i=0;i<rows;i+=16)
{
for (j=0;j<cols;j++)
{
const float * restrict w;
float * restrict y;
float xj;
w = &weights[j*col_stride + i];
xj = x[j];
y = &out[i];
y[0] += w[0]*xj;
y[1] += w[1]*xj;
y[2] += w[2]*xj;
y[3] += w[3]*xj;
y[4] += w[4]*xj;
y[5] += w[5]*xj;
y[6] += w[6]*xj;
y[7] += w[7]*xj;
y[8] += w[8]*xj;
y[9] += w[9]*xj;
y[10] += w[10]*xj;
y[11] += w[11]*xj;
y[12] += w[12]*xj;
y[13] += w[13]*xj;
y[14] += w[14]*xj;
y[15] += w[15]*xj;
}
}
}
static inline void sgemv8x1(float *out, const float *weights, int rows, int cols, int col_stride, const float *x)
{
int i, j;
RNN_CLEAR(out, rows);
for (i=0;i<rows;i+=8)
{
for (j=0;j<cols;j++)
{
const float * restrict w;
float * restrict y;
float xj;
w = &weights[j*col_stride + i];
xj = x[j];
y = &out[i];
y[0] += w[0]*xj;
y[1] += w[1]*xj;
y[2] += w[2]*xj;
y[3] += w[3]*xj;
y[4] += w[4]*xj;
y[5] += w[5]*xj;
y[6] += w[6]*xj;
y[7] += w[7]*xj;
}
}
}
static inline void sgemv(float *out, const float *weights, int rows, int cols, int col_stride, const float *x)
{
if ((rows&0xf) == 0) sgemv16x1(out, weights, rows, cols, col_stride, x);
else if ((rows&0x7) == 0) sgemv8x1(out, weights, rows, cols, col_stride, x);
else {
int i, j;
for (i=0;i<rows;i++)
{
out[i] = 0;
for (j=0;j<cols;j++) out[i] += weights[j*col_stride + i]*x[j];
}
}
}
static inline void sparse_sgemv8x4(float *out, const float *w, const int *idx, int rows, const float *x)
{
int i, j;
RNN_CLEAR(out, rows);
for (i=0;i<rows;i+=8)
{
int cols;
cols = *idx++;
for (j=0;j<cols;j++)
{
int pos;
float * restrict y;
float xj0, xj1, xj2, xj3;
pos = (*idx++);
xj0 = x[pos+0];
xj1 = x[pos+1];
xj2 = x[pos+2];
xj3 = x[pos+3];
y = &out[i];
y[0] += w[0]*xj0;
y[1] += w[1]*xj0;
y[2] += w[2]*xj0;
y[3] += w[3]*xj0;
y[4] += w[4]*xj0;
y[5] += w[5]*xj0;
y[6] += w[6]*xj0;
y[7] += w[7]*xj0;
y[0] += w[8]*xj1;
y[1] += w[9]*xj1;
y[2] += w[10]*xj1;
y[3] += w[11]*xj1;
y[4] += w[12]*xj1;
y[5] += w[13]*xj1;
y[6] += w[14]*xj1;
y[7] += w[15]*xj1;
y[0] += w[16]*xj2;
y[1] += w[17]*xj2;
y[2] += w[18]*xj2;
y[3] += w[19]*xj2;
y[4] += w[20]*xj2;
y[5] += w[21]*xj2;
y[6] += w[22]*xj2;
y[7] += w[23]*xj2;
y[0] += w[24]*xj3;
y[1] += w[25]*xj3;
y[2] += w[26]*xj3;
y[3] += w[27]*xj3;
y[4] += w[28]*xj3;
y[5] += w[29]*xj3;
y[6] += w[30]*xj3;
y[7] += w[31]*xj3;
w += 32;
}
}
}
#ifdef USE_SU_BIAS
static inline void sparse_cgemv8x4(float *out, const opus_int8 *w, const int *idx, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
unsigned char x[MAX_INPUTS];
for (i=0;i<rows;i++) out[i] = 0;
for (i=0;i<cols;i++) x[i] = 127+floor(.5+127*_x[i]);
for (i=0;i<rows;i+=8)
{
int colblocks;
colblocks = *idx++;
for (j=0;j<colblocks;j++)
{
int pos;
float * restrict y;
int xj0, xj1, xj2, xj3;
pos = (*idx++);
xj0 = x[pos+0];
xj1 = x[pos+1];
xj2 = x[pos+2];
xj3 = x[pos+3];
y = &out[i];
y[0] += (w[0]*xj0+w[1]*xj1+w[2]*xj2+w[3]*xj3);
y[1] += (w[4]*xj0+w[5]*xj1+w[6]*xj2+w[7]*xj3);
y[2] += (w[8]*xj0+w[9]*xj1+w[10]*xj2+w[11]*xj3);
y[3] += (w[12]*xj0+w[13]*xj1+w[14]*xj2+w[15]*xj3);
y[4] += (w[16]*xj0+w[17]*xj1+w[18]*xj2+w[19]*xj3);
y[5] += (w[20]*xj0+w[21]*xj1+w[22]*xj2+w[23]*xj3);
y[6] += (w[24]*xj0+w[25]*xj1+w[26]*xj2+w[27]*xj3);
y[7] += (w[28]*xj0+w[29]*xj1+w[30]*xj2+w[31]*xj3);
w += 32;
}
}
for (i=0;i<rows;i++) out[i] *= scale[i];
}
static inline void cgemv8x4(float *out, const opus_int8 *w, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
unsigned char x[MAX_INPUTS];
for (i=0;i<rows;i++) out[i] = 0;
for (i=0;i<cols;i++) x[i] = 127+(int)floor(.5+127*_x[i]);
for (i=0;i<rows;i+=8)
{
for (j=0;j<cols;j+=4)
{
float *y;
float xj0, xj1, xj2, xj3;
xj0 = x[j+0];
xj1 = x[j+1];
xj2 = x[j+2];
xj3 = x[j+3];
y = &out[i];
y[0] += (w[0]*xj0+w[1]*xj1+w[2]*xj2+w[3]*xj3);
y[1] += (w[4]*xj0+w[5]*xj1+w[6]*xj2+w[7]*xj3);
y[2] += (w[8]*xj0+w[9]*xj1+w[10]*xj2+w[11]*xj3);
y[3] += (w[12]*xj0+w[13]*xj1+w[14]*xj2+w[15]*xj3);
y[4] += (w[16]*xj0+w[17]*xj1+w[18]*xj2+w[19]*xj3);
y[5] += (w[20]*xj0+w[21]*xj1+w[22]*xj2+w[23]*xj3);
y[6] += (w[24]*xj0+w[25]*xj1+w[26]*xj2+w[27]*xj3);
y[7] += (w[28]*xj0+w[29]*xj1+w[30]*xj2+w[31]*xj3);
w += 32;
}
}
for (i=0;i<rows;i++) out[i] *= scale[i];
}
#else
static inline void sparse_cgemv8x4(float *out, const opus_int8 *w, const int *idx, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
opus_int8 x[MAX_INPUTS];
for (i=0;i<rows;i++) out[i] = 0;
for (i=0;i<cols;i++) x[i] = (int)floor(.5+127*_x[i]);
for (i=0;i<rows;i+=8)
{
int colblocks;
colblocks = *idx++;
for (j=0;j<colblocks;j++)
{
int pos;
float * restrict y;
int xj0, xj1, xj2, xj3;
pos = (*idx++);
xj0 = x[pos+0];
xj1 = x[pos+1];
xj2 = x[pos+2];
xj3 = x[pos+3];
y = &out[i];
y[0] += (w[0]*xj0+w[1]*xj1+w[2]*xj2+w[3]*xj3);
y[1] += (w[4]*xj0+w[5]*xj1+w[6]*xj2+w[7]*xj3);
y[2] += (w[8]*xj0+w[9]*xj1+w[10]*xj2+w[11]*xj3);
y[3] += (w[12]*xj0+w[13]*xj1+w[14]*xj2+w[15]*xj3);
y[4] += (w[16]*xj0+w[17]*xj1+w[18]*xj2+w[19]*xj3);
y[5] += (w[20]*xj0+w[21]*xj1+w[22]*xj2+w[23]*xj3);
y[6] += (w[24]*xj0+w[25]*xj1+w[26]*xj2+w[27]*xj3);
y[7] += (w[28]*xj0+w[29]*xj1+w[30]*xj2+w[31]*xj3);
w += 32;
}
}
for (i=0;i<rows;i++) out[i] *= scale[i];
}
static inline void cgemv8x4(float *out, const opus_int8 *w, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
opus_int8 x[MAX_INPUTS];
for (i=0;i<rows;i++) out[i] = 0;
for (i=0;i<cols;i++) x[i] = (int)floor(.5+127*_x[i]);
for (i=0;i<rows;i+=8)
{
for (j=0;j<cols;j+=4)
{
float *y;
float xj0, xj1, xj2, xj3;
xj0 = x[j+0];
xj1 = x[j+1];
xj2 = x[j+2];
xj3 = x[j+3];
y = &out[i];
y[0] += (w[0]*xj0+w[1]*xj1+w[2]*xj2+w[3]*xj3);
y[1] += (w[4]*xj0+w[5]*xj1+w[6]*xj2+w[7]*xj3);
y[2] += (w[8]*xj0+w[9]*xj1+w[10]*xj2+w[11]*xj3);
y[3] += (w[12]*xj0+w[13]*xj1+w[14]*xj2+w[15]*xj3);
y[4] += (w[16]*xj0+w[17]*xj1+w[18]*xj2+w[19]*xj3);
y[5] += (w[20]*xj0+w[21]*xj1+w[22]*xj2+w[23]*xj3);
y[6] += (w[24]*xj0+w[25]*xj1+w[26]*xj2+w[27]*xj3);
y[7] += (w[28]*xj0+w[29]*xj1+w[30]*xj2+w[31]*xj3);
w += 32;
}
}
for (i=0;i<rows;i++) out[i] *= scale[i];
}
#endif
/* No AVX2/FMA support */
#ifndef LPCNET_TEST
static inline float lpcnet_exp2(float x)
{
int integer;
float frac;
union {
float f;
opus_uint32 i;
} res;
integer = floor(x);
if (integer < -50)
return 0;
frac = x-integer;
/* K0 = 1, K1 = log(2), K2 = 3-4*log(2), K3 = 3*log(2) - 2 */
res.f = 0.99992522f + frac * (0.69583354f
+ frac * (0.22606716f + 0.078024523f*frac));
res.i = (res.i + (integer<<23)) & 0x7fffffff;
return res.f;
}
#define lpcnet_exp(x) lpcnet_exp2((x)*1.44269504f)
#define fmadd(a, b, c) ((a)*(b)+(c))
static OPUS_INLINE float tanh_approx(float x)
{
const float N0 = 952.52801514f;
const float N1 = 96.39235687f;
const float N2 = 0.60863042f;
const float D0 = 952.72399902f;
const float D1 = 413.36801147f;
const float D2 = 11.88600922f;
float X2, num, den;
X2 = x*x;
num = fmadd(fmadd(N2, X2, N1), X2, N0);
den = fmadd(fmadd(D2, X2, D1), X2, D0);
num = num*x/den;
return MAX32(-1.f, MIN32(1.f, num));
}
static inline float sigmoid_approx(float x)
{
return .5f + .5f*tanh_approx(.5f*x);
}
static inline void softmax(float *y, const float *x, int N)
{
int i;
for (i=0;i<N;i++)
y[i] = lpcnet_exp(x[i]);
}
static inline void vec_tanh(float *y, const float *x, int N)
{
int i;
for (i=0;i<N;i++)
{
y[i] = tanh_approx(x[i]);
}
}
static inline void vec_sigmoid(float *y, const float *x, int N)
{
int i;
for (i=0;i<N;i++)
{
y[i] = sigmoid_approx(x[i]);
}
}
#endif
#define SCALE (128.f*127.f)
#define SCALE_1 (1.f/128.f/127.f)
#endif /*no optimizations*/
#endif /*VEC_H*/

884
third_party/rnnoise/src/vec_avx.h vendored Normal file
View File

@@ -0,0 +1,884 @@
/* Copyright (c) 2018 Mozilla
2012-2017 Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
AVX implementation of vector operations, compile with -mavx
AVX2/FMA implementation of vector operations, compile with -mavx2 -mfma
*/
#ifndef VEC_AVX_H
#define VEC_AVX_H
#include <immintrin.h>
#include <math.h>
#include "x86/x86cpu.h"
#define MAX_INPUTS (2048)
#define USE_SU_BIAS
#ifndef __SSE_4_1__
static inline __m128 mm_floor_ps(__m128 x) {
__m128 half = _mm_set1_ps(0.5);
return _mm_cvtepi32_ps(_mm_cvtps_epi32(_mm_sub_ps(x, half)));
}
#undef _mm_floor_ps
#define _mm_floor_ps(x) mm_floor_ps(x)
#endif
/* If we don't have AVX available, emulate what we need with SSE up to 4.1. */
#ifndef __AVX__
typedef struct {
__m128 lo;
__m128 hi;
} mm256_emu;
#define __m256 mm256_emu
static inline mm256_emu mm256_loadu_ps(const float *src) {
mm256_emu ret;
ret.lo = _mm_loadu_ps(&src[0]);
ret.hi = _mm_loadu_ps(&src[4]);
return ret;
}
#define _mm256_loadu_ps(src) mm256_loadu_ps(src)
static inline void mm256_storeu_ps(float *dst, mm256_emu src) {
_mm_storeu_ps(dst, src.lo);
_mm_storeu_ps(&dst[4], src.hi);
}
#define _mm256_storeu_ps(dst, src) mm256_storeu_ps(dst, src)
static inline mm256_emu mm256_setzero_ps(void) {
mm256_emu ret;
ret.lo = _mm_setzero_ps();
ret.hi = ret.lo;
return ret;
}
#define _mm256_setzero_ps mm256_setzero_ps
static inline mm256_emu mm256_broadcast_ss(const float *x) {
mm256_emu ret;
ret.lo = _mm_set1_ps(*x);
ret.hi = ret.lo;
return ret;
}
#define _mm256_broadcast_ss(x) mm256_broadcast_ss(x)
static inline mm256_emu mm256_set1_ps(float x) {
mm256_emu ret;
ret.lo = _mm_set1_ps(x);
ret.hi = ret.lo;
return ret;
}
#define _mm256_set1_ps(x) mm256_set1_ps(x)
static inline mm256_emu mm256_mul_ps(mm256_emu a, mm256_emu b) {
mm256_emu ret;
ret.lo = _mm_mul_ps(a.lo, b.lo);
ret.hi = _mm_mul_ps(a.hi, b.hi);
return ret;
}
#define _mm256_mul_ps(a,b) mm256_mul_ps(a,b)
static inline mm256_emu mm256_add_ps(mm256_emu a, mm256_emu b) {
mm256_emu ret;
ret.lo = _mm_add_ps(a.lo, b.lo);
ret.hi = _mm_add_ps(a.hi, b.hi);
return ret;
}
#define _mm256_add_ps(a,b) mm256_add_ps(a,b)
static inline mm256_emu mm256_max_ps(mm256_emu a, mm256_emu b) {
mm256_emu ret;
ret.lo = _mm_max_ps(a.lo, b.lo);
ret.hi = _mm_max_ps(a.hi, b.hi);
return ret;
}
#define _mm256_max_ps(a,b) mm256_max_ps(a,b)
static inline mm256_emu mm256_min_ps(mm256_emu a, mm256_emu b) {
mm256_emu ret;
ret.lo = _mm_min_ps(a.lo, b.lo);
ret.hi = _mm_min_ps(a.hi, b.hi);
return ret;
}
#define _mm256_min_ps(a,b) mm256_min_ps(a,b)
static inline mm256_emu mm256_rcp_ps(mm256_emu a) {
mm256_emu ret;
ret.lo = _mm_rcp_ps(a.lo);
ret.hi = _mm_rcp_ps(a.hi);
return ret;
}
#define _mm256_rcp_ps(a) mm256_rcp_ps(a)
static inline __m128 mm256_extractf128_ps(mm256_emu x, int i) {
return (i==0) ? x.lo : x.hi;
}
#undef _mm256_extractf128_ps
#define _mm256_extractf128_ps(x,i) mm256_extractf128_ps(x,i)
static inline mm256_emu mm256_insertf128_ps(mm256_emu dst, __m128 src, int i) {
if (i==0) dst.lo = src;
else dst.hi = src;
return dst;
}
#undef _mm256_insertf128_ps
#define _mm256_insertf128_ps(dst,src,i) mm256_insertf128_ps(dst,src,i)
#endif /* __AVX__ */
/* If we don't have AVX2 available, emulate what we need with SSE up to 4.1. */
#ifndef __AVX2__
typedef struct {
__m128i lo;
__m128i hi;
} mm256i_emu;
typedef __m256i real_m256i;
#define __m256i mm256i_emu
static inline mm256i_emu mm256_setzero_si256(void) {
mm256i_emu ret;
ret.lo = _mm_setzero_si128();
ret.hi = ret.lo;
return ret;
}
#define _mm256_setzero_si256 mm256_setzero_si256
static inline mm256i_emu mm256_loadu_si256(const mm256i_emu *src) {
mm256i_emu ret;
ret.lo = _mm_loadu_si128((const __m128i*)src);
ret.hi = _mm_loadu_si128(&((const __m128i*)src)[1]);
return ret;
}
#define _mm256_loadu_si256(src) mm256_loadu_si256(src)
static inline void mm256_storeu_si256(mm256i_emu *dst, mm256i_emu src) {
_mm_storeu_si128((__m128i*)dst, src.lo);
_mm_storeu_si128(&((__m128i*)dst)[1], src.hi);
}
#define _mm256_storeu_si256(dst, src) mm256_storeu_si256(dst, src)
static inline mm256i_emu mm256_broadcastd_epi32(__m128i x) {
mm256i_emu ret;
ret.hi = ret.lo = _mm_shuffle_epi32(x, 0);
return ret;
}
#define _mm256_broadcastd_epi32(x) mm256_broadcastd_epi32(x)
static inline mm256i_emu mm256_set1_epi32(int x) {
mm256i_emu ret;
ret.lo = _mm_set1_epi32(x);
ret.hi = ret.lo;
return ret;
}
#define _mm256_set1_epi32(x) mm256_set1_epi32(x)
static inline mm256i_emu mm256_set1_epi16(int x) {
mm256i_emu ret;
ret.lo = _mm_set1_epi16(x);
ret.hi = ret.lo;
return ret;
}
#define _mm256_set1_epi16(x) mm256_set1_epi16(x)
static inline mm256i_emu mm256_add_epi32(mm256i_emu a, mm256i_emu b) {
mm256i_emu ret;
ret.lo = _mm_add_epi32(a.lo, b.lo);
ret.hi = _mm_add_epi32(a.hi, b.hi);
return ret;
}
#define _mm256_add_epi32(a,b) mm256_add_epi32(a,b)
static inline mm256i_emu mm256_madd_epi16(mm256i_emu a, mm256i_emu b) {
mm256i_emu ret;
ret.lo = _mm_madd_epi16(a.lo, b.lo);
ret.hi = _mm_madd_epi16(a.hi, b.hi);
return ret;
}
#define _mm256_madd_epi16(a,b) mm256_madd_epi16(a,b)
static inline mm256i_emu mm256_maddubs_epi16(mm256i_emu a, mm256i_emu b) {
mm256i_emu ret;
ret.lo = _mm_maddubs_epi16(a.lo, b.lo);
ret.hi = _mm_maddubs_epi16(a.hi, b.hi);
return ret;
}
#define _mm256_maddubs_epi16(a,b) mm256_maddubs_epi16(a,b)
/* Emulating the conversion functions is tricky because they use __m256i but are defined in AVX.
So we need to make a special when only AVX is available. */
#ifdef __AVX__
typedef union {
mm256i_emu fake;
real_m256i real;
} mm256_union;
static inline __m256 mm256_cvtepi32_ps(mm256i_emu a) {
mm256_union src;
src.fake = a;
return _mm256_cvtepi32_ps(src.real);
}
#define _mm256_cvtepi32_ps(a) mm256_cvtepi32_ps(a)
static inline mm256i_emu mm256_cvtps_epi32(__m256 a) {
mm256_union ret;
ret.real = _mm256_cvtps_epi32(a);
return ret.fake;
}
#define _mm256_cvtps_epi32(a) mm256_cvtps_epi32(a)
#else
static inline mm256_emu mm256_cvtepi32_ps(mm256i_emu a) {
mm256_emu ret;
ret.lo = _mm_cvtepi32_ps(a.lo);
ret.hi = _mm_cvtepi32_ps(a.hi);
return ret;
}
#define _mm256_cvtepi32_ps(a) mm256_cvtepi32_ps(a)
static inline mm256i_emu mm256_cvtps_epi32(mm256_emu a) {
mm256i_emu ret;
ret.lo = _mm_cvtps_epi32(a.lo);
ret.hi = _mm_cvtps_epi32(a.hi);
return ret;
}
#define _mm256_cvtps_epi32(a) mm256_cvtps_epi32(a)
#endif /* __AVX__ */
#endif /* __AVX2__ */
/* In case we don't have FMA, make it a mul and an add. */
#if !(defined(__FMA__) && defined(__AVX__))
#define _mm256_fmadd_ps(a,b,c) _mm256_add_ps(_mm256_mul_ps(a, b), c)
#define _mm_fmadd_ps(a,b,c) _mm_add_ps(_mm_mul_ps(a, b), c)
#endif
#ifdef __AVX2__
static inline __m256 exp8_approx(__m256 X)
{
const __m256 K0 = _mm256_set1_ps(0.99992522f);
const __m256 K1 = _mm256_set1_ps(0.69583354f);
const __m256 K2 = _mm256_set1_ps(0.22606716f);
const __m256 K3 = _mm256_set1_ps(0.078024523f);
const __m256 log2_E = _mm256_set1_ps(1.44269504f);
const __m256 max_in = _mm256_set1_ps(50.f);
const __m256 min_in = _mm256_set1_ps(-50.f);
__m256 XF, Y;
__m256i I;
X = _mm256_mul_ps(X, log2_E);
X = _mm256_max_ps(min_in, _mm256_min_ps(max_in, X));
XF = _mm256_floor_ps(X);
I = _mm256_cvtps_epi32(XF);
X = _mm256_sub_ps(X, XF);
Y = _mm256_fmadd_ps(_mm256_fmadd_ps(_mm256_fmadd_ps(K3, X, K2), X, K1), X, K0);
I = _mm256_slli_epi32(I, 23);
Y = _mm256_castsi256_ps(_mm256_add_epi32(I, _mm256_castps_si256(Y)));
return Y;
}
static inline void vector_ps_to_epi8(unsigned char *x, const float *_x, int len) {
int i;
__m256 const127 = _mm256_set1_ps(127.f);
for (i=0;i<len;i+=8) {
__m256 xf;
__m256i xi;
xf = _mm256_loadu_ps(&_x[i]);
xf = _mm256_fmadd_ps(xf, const127, const127);
xi = _mm256_cvtps_epi32(xf);
xi = _mm256_packus_epi32(xi, _mm256_setzero_si256());
xi = _mm256_permute4x64_epi64(xi, 0xD8);
xi = _mm256_packus_epi16(xi, _mm256_setzero_si256());
xi = _mm256_permutevar8x32_epi32(xi, _mm256_setr_epi32(0,1, 0,0, 0,0, 0,0));
_mm256_storeu_si256 ((__m256i *)(void*)&x[i], xi);
}
}
#else
static inline __m128 exp4_approx(__m128 X)
{
const __m128 K0 = _mm_set1_ps(0.99992522f);
const __m128 K1 = _mm_set1_ps(0.69583354f);
const __m128 K2 = _mm_set1_ps(0.22606716f);
const __m128 K3 = _mm_set1_ps(0.078024523f);
const __m128 log2_E = _mm_set1_ps(1.44269504);
const __m128 max_in = _mm_set1_ps(50.f);
const __m128 min_in = _mm_set1_ps(-50.f);
const __m128i mask = _mm_set1_epi32(0x7fffffff);
__m128 XF, Y;
__m128i I;
X = _mm_mul_ps(X, log2_E);
X = _mm_max_ps(min_in, _mm_min_ps(max_in, X));
XF = _mm_floor_ps(X);
I = _mm_cvtps_epi32(XF);
X = _mm_sub_ps(X, XF);
Y = _mm_fmadd_ps(_mm_fmadd_ps(_mm_fmadd_ps(K3, X, K2), X, K1), X, K0);
I = _mm_slli_epi32(I, 23);
Y = _mm_castsi128_ps(_mm_and_si128(mask, _mm_add_epi32(I, _mm_castps_si128(Y))));
return Y;
}
static inline __m256 exp8_approx(__m256 X)
{
__m256 Y;
__m128 Xhi, Xlo, Yhi, Ylo;
Xhi = _mm256_extractf128_ps(X, 1);
Xlo = _mm256_extractf128_ps(X, 0);
Yhi = exp4_approx(Xhi);
Ylo = exp4_approx(Xlo);
Y = _mm256_insertf128_ps(_mm256_setzero_ps(), Yhi, 1);
Y = _mm256_insertf128_ps(Y, Ylo, 0);
return Y;
}
static inline void vector_ps_to_epi8(unsigned char *x, const float *_x, int len) {
int i;
for (i=0;i<len;i++) x[i] = 127+(int)floor(.5+127*_x[i]);
}
#endif
#ifdef __AVX__
/* Approximating tanh() using a Padé-like rational function:
tanh(x) ~= x * (N0 + N1*x^2 + N2*x^4)/(D0 + D1*x^2 + D2*x^4)
subject to the +/- 1 bounds.
The coefficients were determined by gradient descent trying to minimize
the maximum deviation over the whole range (this is only possible because
of the bounds). The max error is around 3e-4 and is dominated by the
reciprocal approximation (the max error of the rational function is
around 6e-5).
*/
static inline __m256 tanh8_approx(__m256 X)
{
const __m256 N0 = _mm256_set1_ps(952.52801514f);
const __m256 N1 = _mm256_set1_ps(96.39235687f);
const __m256 N2 = _mm256_set1_ps(0.60863042f);
const __m256 D0 = _mm256_set1_ps(952.72399902f);
const __m256 D1 = _mm256_set1_ps(413.36801147f);
const __m256 D2 = _mm256_set1_ps(11.88600922f);
const __m256 max_out = _mm256_set1_ps(1.f);
const __m256 min_out = _mm256_set1_ps(-1.f);
__m256 X2, num, den;
X2 = _mm256_mul_ps(X, X);
num = _mm256_fmadd_ps(_mm256_fmadd_ps(N2, X2, N1), X2, N0);
den = _mm256_fmadd_ps(_mm256_fmadd_ps(D2, X2, D1), X2, D0);
num = _mm256_mul_ps(num, X);
den = _mm256_rcp_ps(den);
num = _mm256_mul_ps(num, den);
return _mm256_max_ps(min_out, _mm256_min_ps(max_out, num));
}
/* Sigmoid approximation using a Padé-like rational function:
1/(1+exp(-x)) ~= 0.5 + x * (N0 + N1*x^2 + N2*x^4)/(D0 + D1*x^2 + D2*x^4)
subject to the [0, 1] bounds.
The coefficients are directly derived by dividing the tanh() coefficients
by powers of two to get the correct scaling. The max error is around 1.5e-4
and is dominated by the reciprocal approximation (the max error of the
rational function is around 3e-5).
*/
static inline __m256 sigmoid8_approx(__m256 X)
{
const __m256 N0 = _mm256_set1_ps(238.13200378f);
const __m256 N1 = _mm256_set1_ps(6.02452230f);
const __m256 N2 = _mm256_set1_ps(0.00950985f);
const __m256 D0 = _mm256_set1_ps(952.72399902f);
const __m256 D1 = _mm256_set1_ps(103.34200287f);
const __m256 D2 = _mm256_set1_ps(0.74287558f);
const __m256 half = _mm256_set1_ps(0.5);
const __m256 max_out = _mm256_set1_ps(1.f);
const __m256 min_out = _mm256_set1_ps(0.f);
__m256 X2, num, den;
X2 = _mm256_mul_ps(X, X);
num = _mm256_fmadd_ps(_mm256_fmadd_ps(N2, X2, N1), X2, N0);
den = _mm256_fmadd_ps(_mm256_fmadd_ps(D2, X2, D1), X2, D0);
num = _mm256_mul_ps(num, X);
den = _mm256_rcp_ps(den);
num = _mm256_fmadd_ps(num, den, half);
return _mm256_max_ps(min_out, _mm256_min_ps(max_out, num));
}
static inline float tanh_approx(float x)
{
float out[8];
__m256 X, Y;
X = _mm256_set1_ps(x);
Y = tanh8_approx(X);
_mm256_storeu_ps(out, Y);
return out[0];
}
static inline float sigmoid_approx(float x)
{
float out[8];
__m256 X, Y;
X = _mm256_set1_ps(x);
Y = sigmoid8_approx(X);
_mm256_storeu_ps(out, Y);
return out[0];
}
#else
static inline __m128 tanh4_approx(__m128 X)
{
const __m128 N0 = _mm_set1_ps(952.52801514f);
const __m128 N1 = _mm_set1_ps(96.39235687f);
const __m128 N2 = _mm_set1_ps(0.60863042f);
const __m128 D0 = _mm_set1_ps(952.72399902f);
const __m128 D1 = _mm_set1_ps(413.36801147f);
const __m128 D2 = _mm_set1_ps(11.88600922f);
const __m128 max_out = _mm_set1_ps(1.f);
const __m128 min_out = _mm_set1_ps(-1.f);
__m128 X2, num, den;
X2 = _mm_mul_ps(X, X);
num = _mm_fmadd_ps(_mm_fmadd_ps(N2, X2, N1), X2, N0);
den = _mm_fmadd_ps(_mm_fmadd_ps(D2, X2, D1), X2, D0);
num = _mm_mul_ps(num, X);
den = _mm_rcp_ps(den);
num = _mm_mul_ps(num, den);
return _mm_max_ps(min_out, _mm_min_ps(max_out, num));
}
static inline __m128 sigmoid4_approx(__m128 X)
{
const __m128 N0 = _mm_set1_ps(238.13200378f);
const __m128 N1 = _mm_set1_ps(6.02452230f);
const __m128 N2 = _mm_set1_ps(0.00950985f);
const __m128 D0 = _mm_set1_ps(952.72399902f);
const __m128 D1 = _mm_set1_ps(103.34200287f);
const __m128 D2 = _mm_set1_ps(0.74287558f);
const __m128 half = _mm_set1_ps(0.5);
const __m128 max_out = _mm_set1_ps(1.f);
const __m128 min_out = _mm_set1_ps(0.f);
__m128 X2, num, den;
X2 = _mm_mul_ps(X, X);
num = _mm_fmadd_ps(_mm_fmadd_ps(N2, X2, N1), X2, N0);
den = _mm_fmadd_ps(_mm_fmadd_ps(D2, X2, D1), X2, D0);
num = _mm_mul_ps(num, X);
den = _mm_rcp_ps(den);
num = _mm_fmadd_ps(num, den, half);
return _mm_max_ps(min_out, _mm_min_ps(max_out, num));
}
static inline float tanh_approx(float x)
{
float out[4];
__m128 X, Y;
X = _mm_set1_ps(x);
Y = tanh4_approx(X);
_mm_storeu_ps(out, Y);
return out[0];
}
static inline float sigmoid_approx(float x)
{
float out[4];
__m128 X, Y;
X = _mm_set1_ps(x);
Y = sigmoid4_approx(X);
_mm_storeu_ps(out, Y);
return out[0];
}
#endif
static inline float lpcnet_exp(float x)
{
float out[8];
__m256 X, Y;
X = _mm256_set1_ps(x);
Y = exp8_approx(X);
_mm256_storeu_ps(out, Y);
return out[0];
}
static inline void softmax(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-7;i+=8)
{
__m256 X, Y;
X = _mm256_loadu_ps(&x[i]);
Y = exp8_approx(X);
_mm256_storeu_ps(&y[i], Y);
}
for (;i<N;i++)
y[i] = lpcnet_exp(x[i]);
}
#ifdef __AVX__
static inline void vec_tanh(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-7;i+=8)
{
__m256 X, Y;
X = _mm256_loadu_ps(&x[i]);
Y = tanh8_approx(X);
_mm256_storeu_ps(&y[i], Y);
}
for (;i<N;i++)
{
y[i] = tanh_approx(x[i]);
}
}
static inline void vec_sigmoid(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-7;i+=8)
{
__m256 X, Y;
X = _mm256_loadu_ps(&x[i]);
Y = sigmoid8_approx(X);
_mm256_storeu_ps(&y[i], Y);
}
for (;i<N;i++)
{
y[i] = sigmoid_approx(x[i]);
}
}
#else
static inline void vec_tanh(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-3;i+=4)
{
__m128 X, Y;
X = _mm_loadu_ps(&x[i]);
Y = tanh4_approx(X);
_mm_storeu_ps(&y[i], Y);
}
for (;i<N;i++)
{
y[i] = tanh_approx(x[i]);
}
}
static inline void vec_sigmoid(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-3;i+=4)
{
__m128 X, Y;
X = _mm_loadu_ps(&x[i]);
Y = sigmoid4_approx(X);
_mm_storeu_ps(&y[i], Y);
}
for (;i<N;i++)
{
y[i] = sigmoid_approx(x[i]);
}
}
#endif
#if defined(__AVXVNNI__) || defined(__AVX512VNNI__)
#define opus_mm256_dpbusds_epi32(src, a, b) _mm256_dpbusds_epi32(src, a, b)
#elif defined(__AVX2__)
static inline __m256i opus_mm256_dpbusds_epi32(__m256i src, __m256i a, __m256i b) {
__m256i ones, tmp;
ones = _mm256_set1_epi16(1);
tmp = _mm256_maddubs_epi16(a, b);
tmp = _mm256_madd_epi16(tmp, ones);
return _mm256_add_epi32(src, tmp);
}
#elif defined(__SSSE3__)
static inline mm256i_emu opus_mm256_dpbusds_epi32(mm256i_emu src, mm256i_emu a, mm256i_emu b) {
mm256i_emu ones, tmp;
ones = _mm256_set1_epi16(1);
tmp = _mm256_maddubs_epi16(a, b);
tmp = _mm256_madd_epi16(tmp, ones);
return _mm256_add_epi32(src, tmp);
}
#elif defined(__SSE2__)
static inline __m128i mm_dpbusds_epi32(__m128i src, __m128i a, __m128i b) {
__m128i ah, al, bh, bl, tmp;
ah = _mm_srli_epi16(a, 8);
bh = _mm_srai_epi16(b, 8);
al = _mm_srli_epi16(_mm_slli_epi16(a, 8), 8);
bl = _mm_srai_epi16(_mm_slli_epi16(b, 8), 8);
tmp = _mm_add_epi32(_mm_madd_epi16(ah, bh), _mm_madd_epi16(al, bl));
return _mm_add_epi32(src, tmp);
}
static inline mm256i_emu opus_mm256_dpbusds_epi32(mm256i_emu src, mm256i_emu a, mm256i_emu b) {
mm256i_emu res;
res.hi = mm_dpbusds_epi32(src.hi, a.hi, b.hi);
res.lo = mm_dpbusds_epi32(src.lo, a.lo, b.lo);
return res;
}
#else
#error "No optimizations in vec_avx.h. This should never happen. "
#endif
static inline void sgemv(float *out, const float *weights, int rows, int cols, int col_stride, const float *x)
{
int i, j;
i=0;
for (;i<rows-15;i+=16)
{
float *y;
__m256 vy0, vy8;
y = &out[i];
vy0 = _mm256_setzero_ps();
vy8 = _mm256_setzero_ps();
for (j=0;j<cols;j++)
{
__m256 vxj;
__m256 vw;
vxj = _mm256_broadcast_ss(&x[j]);
vw = _mm256_loadu_ps(&weights[j*col_stride + i]);
vy0 = _mm256_fmadd_ps(vw, vxj, vy0);
vw = _mm256_loadu_ps(&weights[j*col_stride + i + 8]);
vy8 = _mm256_fmadd_ps(vw, vxj, vy8);
}
_mm256_storeu_ps (&y[0], vy0);
_mm256_storeu_ps (&y[8], vy8);
}
for (;i<rows-7;i+=8)
{
float *y;
__m256 vy0;
y = &out[i];
vy0 = _mm256_setzero_ps();
for (j=0;j<cols;j++)
{
__m256 vxj;
__m256 vw;
vxj = _mm256_broadcast_ss(&x[j]);
vw = _mm256_loadu_ps(&weights[j*col_stride + i]);
vy0 = _mm256_fmadd_ps(vw, vxj, vy0);
}
_mm256_storeu_ps (&y[0], vy0);
}
for (;i<rows-3;i+=4)
{
float *y;
__m128 vy0;
y = &out[i];
vy0 = _mm_setzero_ps();
for (j=0;j<cols;j++)
{
__m128 vxj;
__m128 vw;
vxj = _mm_set1_ps(x[j]);
vw = _mm_loadu_ps(&weights[j*col_stride + i]);
vy0 = _mm_fmadd_ps(vw, vxj, vy0);
}
_mm_storeu_ps (&y[0], vy0);
}
for (;i<rows;i++)
{
out[i] = 0;
for (j=0;j<cols;j++) out[i] += weights[j*col_stride + i]*x[j];
}
}
static inline void sparse_sgemv8x4(float *out, const float *weights, const int *idx, int rows, const float *x)
{
int i, j;
for (i=0;i<rows;i+=8)
{
float *y;
int cols;
__m256 vy0;
y = &out[i];
vy0 = _mm256_setzero_ps();
cols = *idx++;
for (j=0;j<cols;j++)
{
int id;
__m256 vxj;
__m256 vw;
id = *idx++;
vxj = _mm256_broadcast_ss(&x[id]);
vw = _mm256_loadu_ps(&weights[0]);
vy0 = _mm256_fmadd_ps(vw, vxj, vy0);
vxj = _mm256_broadcast_ss(&x[id+1]);
vw = _mm256_loadu_ps(&weights[8]);
vy0 = _mm256_fmadd_ps(vw, vxj, vy0);
vxj = _mm256_broadcast_ss(&x[id+2]);
vw = _mm256_loadu_ps(&weights[16]);
vy0 = _mm256_fmadd_ps(vw, vxj, vy0);
vxj = _mm256_broadcast_ss(&x[id+3]);
vw = _mm256_loadu_ps(&weights[24]);
vy0 = _mm256_fmadd_ps(vw, vxj, vy0);
weights += 32;
}
_mm256_storeu_ps (&y[0], vy0);
}
}
static inline void sparse_cgemv8x4(float *_out, const opus_int8 *w, const int *idx, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
unsigned char x[MAX_INPUTS];
/*for (i=0;i<cols;i++) x[i] = 127+floor(.5+127*_x[i]);*/
vector_ps_to_epi8(x, _x, cols);
for (i=0;i<rows;i+=8)
{
int colblocks;
__m256i vy0;
__m256 vout;
colblocks = *idx++;
vy0 = _mm256_setzero_si256();
j=0;
#if 1 /* Unrolling by 4 gives some gain, comment out if it does not. */
for (;j<colblocks-3;j+=4)
{
__m256i vxj;
__m256i vw;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[*idx++]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[*idx++]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[*idx++]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[*idx++]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
}
#endif
for (;j<colblocks;j++)
{
__m256i vxj;
__m256i vw;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[*idx++]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
}
vout = _mm256_cvtepi32_ps(vy0);
vout = _mm256_mul_ps(vout, _mm256_loadu_ps(&scale[i]));
_mm256_storeu_ps(&_out[i], vout);
}
}
static inline void cgemv8x4(float *_out, const opus_int8 *w, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
unsigned char x[MAX_INPUTS];
/*for (i=0;i<cols;i++) x[i] = 127+floor(.5+127*_x[i]);*/
vector_ps_to_epi8(x, _x, cols);
for (i=0;i<rows;i+=8)
{
__m256i vy0;
__m256 vout;
vy0 = _mm256_setzero_si256();
j=0;
#if 1 /* Unrolling by 4 gives some gain, comment out if it does not. */
for (;j<cols-12;j+=16)
{
__m256i vxj;
__m256i vw;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[j]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[j+4]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[j+8]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[j+12]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
}
#endif
for (;j<cols;j+=4)
{
__m256i vxj;
__m256i vw;
vxj = _mm256_broadcastd_epi32(_mm_loadu_si32(&x[j]));
vw = _mm256_loadu_si256((const __m256i *)(void*)w);
vy0 = opus_mm256_dpbusds_epi32(vy0, vxj, vw);
w += 32;
}
vout = _mm256_cvtepi32_ps(vy0);
vout = _mm256_mul_ps(vout, _mm256_loadu_ps(&scale[i]));
_mm256_storeu_ps(&_out[i], vout);
}
}
#define SCALE (128.f*127.f)
#define SCALE_1 (1.f/128.f/127.f)
#define USE_SU_BIAS
#endif /*VEC_AVX_H*/

474
third_party/rnnoise/src/vec_neon.h vendored Normal file
View File

@@ -0,0 +1,474 @@
/* Copyright (c) 2018 David Rowe
2018 Mozilla
2008-2011 Octasic Inc.
2012-2017 Jean-Marc Valin */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* NEON support for ARM machines */
#ifndef VEC_NEON_H
#define VEC_NEON_H
#include <arm_neon.h>
#include "opus_types.h"
#include "common.h"
#if defined(__arm__) && !defined(__aarch64__) && (__ARM_ARCH < 8 || !defined(__clang__))
/* Emulate vcvtnq_s32_f32() for ARMv7 Neon. */
static OPUS_INLINE int32x4_t vcvtnq_s32_f32(float32x4_t x) {
return vrshrq_n_s32(vcvtq_n_s32_f32(x, 8), 8);
}
static OPUS_INLINE int16x8_t vpaddq_s16(int16x8_t a, int16x8_t b) {
return vcombine_s16(vpadd_s16(vget_low_s16(a), vget_high_s16(a)), vpadd_s16(vget_low_s16(b), vget_high_s16(b)));
}
static OPUS_INLINE int16x8_t vmull_high_s8(int8x16_t a, int8x16_t b) {
return vmull_s8(vget_high_s8(a), vget_high_s8(b));
}
#endif
#ifdef __ARM_FEATURE_FMA
/* If we can, force the compiler to use an FMA instruction rather than break
vmlaq_f32() into fmul/fadd. */
#define vmlaq_f32(a,b,c) vfmaq_f32(a,b,c)
#endif
#ifndef LPCNET_TEST
static inline float32x4_t exp4_approx(float32x4_t x) {
int32x4_t i;
float32x4_t xf;
x = vmaxq_f32(vminq_f32(x, vdupq_n_f32(88.f)), vdupq_n_f32(-88.f));
/* express exp(x) as exp2(x/log(2)), add 127 for the exponent later */
x = vmlaq_f32(vdupq_n_f32(127.f), x, vdupq_n_f32(1.44269504f));
/* split into integer and fractional parts */
i = vcvtq_s32_f32(x);
xf = vcvtq_f32_s32(i);
x = vsubq_f32(x, xf);
float32x4_t K0 = vdupq_n_f32(0.99992522f);
float32x4_t K1 = vdupq_n_f32(0.69583354f);
float32x4_t K2 = vdupq_n_f32(0.22606716f);
float32x4_t K3 = vdupq_n_f32(0.078024523f);
float32x4_t Y = vmlaq_f32(K0, x, vmlaq_f32(K1, x, vmlaq_f32(K2, K3, x)));
/* compute 2^i */
float32x4_t exponent = vreinterpretq_f32_s32(vshlq_n_s32(i, 23));
Y = vmulq_f32(Y, exponent);
return Y;
}
static inline float32x4_t tanh4_approx(float32x4_t X)
{
const float32x4_t N0 = vdupq_n_f32(952.52801514f);
const float32x4_t N1 = vdupq_n_f32(96.39235687f);
const float32x4_t N2 = vdupq_n_f32(0.60863042f);
const float32x4_t D0 = vdupq_n_f32(952.72399902f);
const float32x4_t D1 = vdupq_n_f32(413.36801147f);
const float32x4_t D2 = vdupq_n_f32(11.88600922f);
const float32x4_t max_out = vdupq_n_f32(1.f);
const float32x4_t min_out = vdupq_n_f32(-1.f);
float32x4_t X2, num, den;
X2 = vmulq_f32(X, X);
num = vmlaq_f32(N0, X2, vmlaq_f32(N1, N2, X2));
den = vmlaq_f32(D0, X2, vmlaq_f32(D1, D2, X2));
num = vmulq_f32(num, X);
den = vrecpeq_f32(den);
num = vmulq_f32(num, den);
return vmaxq_f32(min_out, vminq_f32(max_out, num));
}
static inline float32x4_t sigmoid4_approx(float32x4_t X)
{
const float32x4_t N0 = vdupq_n_f32(238.13200378f);
const float32x4_t N1 = vdupq_n_f32(6.02452230f);
const float32x4_t N2 = vdupq_n_f32(0.00950985f);
const float32x4_t D0 = vdupq_n_f32(952.72399902f);
const float32x4_t D1 = vdupq_n_f32(103.34200287f);
const float32x4_t D2 = vdupq_n_f32(0.74287558f);
const float32x4_t half = vdupq_n_f32(0.5f);
const float32x4_t max_out = vdupq_n_f32(1.f);
const float32x4_t min_out = vdupq_n_f32(0.f);
float32x4_t X2, num, den;
X2 = vmulq_f32(X, X);
num = vmlaq_f32(N0, X2, vmlaq_f32(N1, N2, X2));
den = vmlaq_f32(D0, X2, vmlaq_f32(D1, D2, X2));
num = vmulq_f32(num, X);
den = vrecpeq_f32(den);
num = vmlaq_f32(half, num, den);
return vmaxq_f32(min_out, vminq_f32(max_out, num));
}
static inline float lpcnet_exp(float x)
{
float out[4];
float32x4_t X, Y;
X = vdupq_n_f32(x);
Y = exp4_approx(X);
vst1q_f32(out, Y);
return out[0];
}
static inline float tanh_approx(float x)
{
float out[4];
float32x4_t X, Y;
X = vdupq_n_f32(x);
Y = tanh4_approx(X);
vst1q_f32(out, Y);
return out[0];
}
static inline float sigmoid_approx(float x)
{
float out[4];
float32x4_t X, Y;
X = vdupq_n_f32(x);
Y = sigmoid4_approx(X);
vst1q_f32(out, Y);
return out[0];
}
static inline void softmax(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-3;i+=4)
{
float32x4_t X, Y;
X = vld1q_f32(&x[i]);
Y = exp4_approx(X);
vst1q_f32(&y[i], Y);
}
for (;i<N;i++)
y[i] = lpcnet_exp(x[i]);
}
static inline void vec_tanh(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-3;i+=4)
{
float32x4_t X, Y;
X = vld1q_f32(&x[i]);
Y = tanh4_approx(X);
vst1q_f32(&y[i], Y);
}
for (;i<N;i++)
{
float ex2;
ex2 = lpcnet_exp(2*x[i]);
y[i] = (ex2-1)/(ex2+1);
}
}
static inline void vec_sigmoid(float *y, const float *x, int N)
{
int i;
for (i=0;i<N-3;i+=4)
{
float32x4_t X, Y;
X = vld1q_f32(&x[i]);
Y = sigmoid4_approx(X);
vst1q_f32(&y[i], Y);
}
for (;i<N;i++)
{
float ex;
ex = lpcnet_exp(x[i]);
y[i] = (ex)/(ex+1);
}
}
#endif
static inline void sgemv16x1(float *out, const float *weights, int rows, int cols, int col_stride, const float *x)
{
int i, j;
for (i=0;i<rows;i+=16)
{
float * restrict y = &out[i];
/* keep y[0..15] in registers for duration of inner loop */
float32x4_t y0_3 = vdupq_n_f32(0);
float32x4_t y4_7 = vdupq_n_f32(0);
float32x4_t y8_11 = vdupq_n_f32(0);
float32x4_t y12_15 = vdupq_n_f32(0);
for (j=0;j<cols;j++)
{
const float * restrict w;
float32x4_t wvec0_3, wvec4_7, wvec8_11, wvec12_15;
float32x4_t xj;
w = &weights[j*col_stride + i];
wvec0_3 = vld1q_f32(&w[0]);
wvec4_7 = vld1q_f32(&w[4]);
wvec8_11 = vld1q_f32(&w[8]);
wvec12_15 = vld1q_f32(&w[12]);
xj = vld1q_dup_f32(&x[j]);
y0_3 = vmlaq_f32(y0_3, wvec0_3, xj);
y4_7 = vmlaq_f32(y4_7, wvec4_7, xj);
y8_11 = vmlaq_f32(y8_11, wvec8_11, xj);
y12_15 = vmlaq_f32(y12_15, wvec12_15, xj);
}
/* save y[0..15] back to memory */
vst1q_f32(&y[0], y0_3);
vst1q_f32(&y[4], y4_7);
vst1q_f32(&y[8], y8_11);
vst1q_f32(&y[12], y12_15);
}
}
static inline void sgemv8x1(float *out, const float *weights, int rows, int cols, int col_stride, const float *x)
{
int i, j;
for (i=0;i<rows;i+=8)
{
float * restrict y = &out[i];
/* keep y[0..15] in registers for duration of inner loop */
float32x4_t y0_3 = vdupq_n_f32(0);
float32x4_t y4_7 = vdupq_n_f32(0);
for (j=0;j<cols;j++)
{
const float * restrict w;
float32x4_t wvec0_3, wvec4_7;
float32x4_t xj;
w = &weights[j*col_stride + i];
wvec0_3 = vld1q_f32(&w[0]);
wvec4_7 = vld1q_f32(&w[4]);
xj = vld1q_dup_f32(&x[j]);
y0_3 = vmlaq_f32(y0_3, wvec0_3, xj);
y4_7 = vmlaq_f32(y4_7, wvec4_7, xj);
}
/* save y[0..15] back to memory */
vst1q_f32(&y[0], y0_3);
vst1q_f32(&y[4], y4_7);
}
}
static inline void sgemv(float *out, const float *weights, int rows, int cols, int col_stride, const float *x)
{
if ((rows&0xf) == 0) sgemv16x1(out, weights, rows, cols, col_stride, x);
else if ((rows&0x7) == 0) sgemv8x1(out, weights, rows, cols, col_stride, x);
else {
int i, j;
for (i=0;i<rows;i++)
{
out[i] = 0;
for (j=0;j<cols;j++) out[i] += weights[j*col_stride + i]*x[j];
}
}
}
/* Temporarily use unoptimized version */
static inline void sparse_sgemv8x4(float *out, const float *w, const int *idx, int rows, const float *x)
{
int i, j;
RNN_CLEAR(out, rows);
for (i=0;i<rows;i+=8)
{
int cols;
cols = *idx++;
for (j=0;j<cols;j++)
{
int pos;
float * restrict y;
float xj0, xj1, xj2, xj3;
pos = (*idx++);
xj0 = x[pos+0];
xj1 = x[pos+1];
xj2 = x[pos+2];
xj3 = x[pos+3];
y = &out[i];
y[0] += w[0]*xj0;
y[1] += w[1]*xj0;
y[2] += w[2]*xj0;
y[3] += w[3]*xj0;
y[4] += w[4]*xj0;
y[5] += w[5]*xj0;
y[6] += w[6]*xj0;
y[7] += w[7]*xj0;
y[0] += w[8]*xj1;
y[1] += w[9]*xj1;
y[2] += w[10]*xj1;
y[3] += w[11]*xj1;
y[4] += w[12]*xj1;
y[5] += w[13]*xj1;
y[6] += w[14]*xj1;
y[7] += w[15]*xj1;
y[0] += w[16]*xj2;
y[1] += w[17]*xj2;
y[2] += w[18]*xj2;
y[3] += w[19]*xj2;
y[4] += w[20]*xj2;
y[5] += w[21]*xj2;
y[6] += w[22]*xj2;
y[7] += w[23]*xj2;
y[0] += w[24]*xj3;
y[1] += w[25]*xj3;
y[2] += w[26]*xj3;
y[3] += w[27]*xj3;
y[4] += w[28]*xj3;
y[5] += w[29]*xj3;
y[6] += w[30]*xj3;
y[7] += w[31]*xj3;
w += 32;
}
}
}
#define SCALE (128.f*127.f)
#define SCALE_1 (1.f/128.f/127.f)
#define MAX_INPUTS 2048
#define MAX_OUTPUTS 8192
#if __ARM_FEATURE_DOTPROD
static inline int32x4_t vdotprod(int32x4_t acc, int8x16_t a, int8x16_t b) {
return vdotq_s32(acc, a, b);
}
#else
static inline int32x4_t vdotprod(int32x4_t acc, int8x16_t a, int8x16_t b)
{
return vpadalq_s16(acc, vpaddq_s16(vmull_s8(vget_low_s8(a), vget_low_s8(b)), vmull_high_s8(a, b)));
}
#endif
static inline void cgemv8x4(float *_out, const opus_int8 *w, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
opus_int32 x_int[MAX_INPUTS/4];
opus_int8 *x = (opus_int8*) x_int;
const float32x4_t const127 = vdupq_n_f32(127.);
for (i=0;i<cols;i+=8) {
int32x4_t xi0, xi4;
int16x8_t x_short;
xi0 = vcvtnq_s32_f32(vmulq_f32(const127, vld1q_f32(&_x[i])));
xi4 = vcvtnq_s32_f32(vmulq_f32(const127, vld1q_f32(&_x[i+4])));
x_short = vcombine_s16(vmovn_s32(xi0), vmovn_s32(xi4));
vst1_s8(&x[i], vmovn_s16(x_short));
}
for (i=0;i<rows;i+=8)
{
int32x4_t acc0, acc1;
int32x4_t acc2, acc3;
acc0 = vdupq_n_s32(0);
acc1 = vdupq_n_s32(0);
acc2 = vdupq_n_s32(0);
acc3 = vdupq_n_s32(0);
j=0;
for (;j<cols-4;j+=8)
{
int8x16_t vw0, vw1, vw2, vw3, vx0, vx1;
vx0 = (int8x16_t)vld1q_dup_s32((int*)(void*)&x[j]);
vw0 = vld1q_s8(w);
vw1 = vld1q_s8(&w[16]);
acc0 = vdotprod(acc0, vw0, vx0);
acc1 = vdotprod(acc1, vw1, vx0);
vx1 = (int8x16_t)vld1q_dup_s32((int*)(void*)&x[j+4]);
vw2 = vld1q_s8(&w[32]);
vw3 = vld1q_s8(&w[48]);
acc2 = vdotprod(acc2, vw2, vx1);
acc3 = vdotprod(acc3, vw3, vx1);
w += 64;
}
acc0 = vaddq_s32(acc0, acc2);
acc1 = vaddq_s32(acc1, acc3);
for (;j<cols;j+=4)
{
int8x16_t vw0, vw1, vx;
vx = (int8x16_t)vld1q_dup_s32((int*)(void*)&x[j]);
vw0 = vld1q_s8(w);
vw1 = vld1q_s8(&w[16]);
acc0 = vdotprod(acc0, vw0, vx);
acc1 = vdotprod(acc1, vw1, vx);
w += 32;
}
vst1q_f32(&_out[i], vmulq_f32(vld1q_f32(&scale[i]), vcvtq_f32_s32(acc0)));
vst1q_f32(&_out[i+4], vmulq_f32(vld1q_f32(&scale[i+4]), vcvtq_f32_s32(acc1)));
}
}
static inline void sparse_cgemv8x4(float *_out, const opus_int8 *w, const int *idx, const float *scale, int rows, int cols, const float *_x)
{
int i, j;
opus_int32 x_int[MAX_INPUTS/4];
opus_int8 *x = (opus_int8*) x_int;
const float32x4_t const127 = vdupq_n_f32(127.);
for (i=0;i<cols;i+=8) {
int32x4_t xi0, xi4;
int16x8_t x_short;
xi0 = vcvtnq_s32_f32(vmulq_f32(const127, vld1q_f32(&_x[i])));
xi4 = vcvtnq_s32_f32(vmulq_f32(const127, vld1q_f32(&_x[i+4])));
x_short = vcombine_s16(vmovn_s32(xi0), vmovn_s32(xi4));
vst1_s8(&x[i], vmovn_s16(x_short));
}
for (i=0;i<rows;i+=8)
{
int colblocks;
int32x4_t acc0, acc1;
acc0 = vdupq_n_s32(0);
acc1 = vdupq_n_s32(0);
colblocks = *idx++;
for (j=0;j<colblocks;j++)
{
int pos;
pos = (*idx++);
int8x16_t vw0, vw1, vx;
vx = (int8x16_t)vld1q_dup_s32((int*)(void*)&x[pos]);
vw0 = vld1q_s8(w);
vw1 = vld1q_s8(&w[16]);
acc0 = vdotprod(acc0, vw0, vx);
acc1 = vdotprod(acc1, vw1, vx);
w += 32;
}
vst1q_f32(&_out[i], vmulq_f32(vld1q_f32(&scale[i]), vcvtq_f32_s32(acc0)));
vst1q_f32(&_out[i+4], vmulq_f32(vld1q_f32(&scale[i+4]), vcvtq_f32_s32(acc1)));
}
}
#endif

85
third_party/rnnoise/src/x86/dnn_x86.h vendored Normal file
View File

@@ -0,0 +1,85 @@
/* Copyright (c) 2011-2019 Mozilla
2023 Amazon */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef DNN_X86_H
#define DNN_X86_H
#include "cpu_support.h"
#include "opus_types.h"
void compute_linear_sse4_1(const LinearLayer *linear, float *out, const float *in);
void compute_activation_sse4_1(float *output, const float *input, int N, int activation);
void compute_conv2d_sse4_1(const Conv2dLayer *conv, float *out, float *mem, const float *in, int height, int hstride, int activation);
void compute_linear_avx2(const LinearLayer *linear, float *out, const float *in);
void compute_activation_avx2(float *output, const float *input, int N, int activation);
void compute_conv2d_avx2(const Conv2dLayer *conv, float *out, float *mem, const float *in, int height, int hstride, int activation);
#ifdef RNN_ENABLE_X86_RTCD
extern void (*const RNN_COMPUTE_LINEAR_IMPL[OPUS_ARCHMASK + 1])(
const LinearLayer *linear,
float *out,
const float *in
);
#define OVERRIDE_COMPUTE_LINEAR
#define compute_linear(linear, out, in, arch) \
((*RNN_COMPUTE_LINEAR_IMPL[(arch) & OPUS_ARCHMASK])(linear, out, in))
extern void (*const RNN_COMPUTE_ACTIVATION_IMPL[OPUS_ARCHMASK + 1])(
float *output,
const float *input,
int N,
int activation
);
#define OVERRIDE_COMPUTE_ACTIVATION
#define compute_activation(output, input, N, activation, arch) \
((*RNN_COMPUTE_ACTIVATION_IMPL[(arch) & OPUS_ARCHMASK])(output, input, N, activation))
extern void (*const RNN_COMPUTE_CONV2D_IMPL[OPUS_ARCHMASK + 1])(
const Conv2dLayer *conv,
float *out,
float *mem,
const float *in,
int height,
int hstride,
int activation
);
#define OVERRIDE_COMPUTE_CONV2D
#define compute_conv2d(conv, out, mem, in, height, hstride, activation, arch) \
((*RNN_COMPUTE_CONV2D_IMPL[(arch) & OPUS_ARCHMASK])(conv, out, mem, in, height, hstride, activation))
#endif
#endif /* DNN_X86_H */

47
third_party/rnnoise/src/x86/x86_arch_macros.h vendored Normal file
View File

@@ -0,0 +1,47 @@
/* Copyright (c) 2023 Amazon */
/*
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef _MSC_VER
# ifdef OPUS_X86_MAY_HAVE_SSE
# ifndef __SSE__
# define __SSE__
# endif
# endif
# ifdef OPUS_X86_MAY_HAVE_SSE2
# ifndef __SSE2__
# define __SSE2__
# endif
# endif
# ifdef OPUS_X86_MAY_HAVE_SSE4_1
# ifndef __SSE4_1__
# define __SSE4_1__
# endif
# endif
#endif

88
third_party/rnnoise/src/x86/x86cpu.h vendored Normal file
View File

@@ -0,0 +1,88 @@
/* Copyright (c) 2014, Cisco Systems, INC
Written by XiangMingZhu WeiZhou MinPeng YanWang
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#if !defined(X86CPU_H)
# define X86CPU_H
# define MAY_HAVE_SSE4_1(name) name ## _sse4_1
# define MAY_HAVE_AVX2(name) name ## _avx2
# ifdef RNN_ENABLE_X86_RTCD
int opus_select_arch(void);
# endif
# if defined(__SSE2__)
# include "common.h"
/*MOVD should not impose any alignment restrictions, but the C standard does,
and UBSan will report errors if we actually make unaligned accesses.
Use this to work around those restrictions (which should hopefully all get
optimized to a single MOVD instruction).
GCC implemented _mm_loadu_si32() since GCC 11; HOWEVER, there is a bug!
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99754
LLVM implemented _mm_loadu_si32() since Clang 8.0, however the
__clang_major__ version number macro is unreliable, as vendors
(specifically, Apple) will use different numbering schemes than upstream.
Clang's advice is "use feature detection", but they do not provide feature
detection support for specific SIMD functions.
We follow the approach from the SIMDe project and instead detect unrelated
features that should be available in the version we want (see
<https://github.com/simd-everywhere/simde/blob/master/simde/simde-detect-clang.h>).*/
# if defined(__clang__)
# if __has_warning("-Wextra-semi-stmt") || \
__has_builtin(__builtin_rotateleft32)
# define OPUS_CLANG_8 (1)
# endif
# endif
# if !defined(_MSC_VER) && !OPUS_GNUC_PREREQ(11,3) && !defined(OPUS_CLANG_8)
# include <string.h>
# include <emmintrin.h>
# ifdef _mm_loadu_si32
# undef _mm_loadu_si32
# endif
# define _mm_loadu_si32 WORKAROUND_mm_loadu_si32
static inline __m128i WORKAROUND_mm_loadu_si32(void const* mem_addr) {
int val;
memcpy(&val, mem_addr, sizeof(val));
return _mm_cvtsi32_si128(val);
}
# elif defined(_MSC_VER)
/* MSVC needs this for _mm_loadu_si32 */
# include <immintrin.h>
# endif
# define OP_CVTEPI8_EPI32_M32(x) \
(_mm_cvtepi8_epi32(_mm_loadu_si32(x)))
# define OP_CVTEPI16_EPI32_M64(x) \
(_mm_cvtepi16_epi32(_mm_loadl_epi64((__m128i *)(void*)(x))))
# endif
#endif