audio: fix saturation nonlinearity in clip_* functions

The current positive limit for the saturation nonlinearity is only correct if the type of the result has 8 bits or less. Signed-off-by: Volker Rümelin <[email protected]> Message-id: [email protected] Signed-off-by: Gerd Hoffmann <[email protected]>
devos50 · Mar 16, 2020 · 194bdf5 · 194bdf5
1 parent 4218fdd
commit 194bdf5
Showing 1 changed file with 2 additions and 3 deletions.
diff --git a/audio/mixeng_template.h b/audio/mixeng_template.h
@@ -83,10 +83,9 @@ static inline int64_t glue (conv_, ET) (IN_T v)
 
 static inline IN_T glue (clip_, ET) (int64_t v)
 {
-    if (v >= 0x7f000000) {
+    if (v >= 0x7fffffffLL) {
         return IN_MAX;
-    }
-    else if (v < -2147483648LL) {
+    } else if (v < -2147483648LL) {
         return IN_MIN;
     }