FDK sanitization fixes

Bug: 110641440
Bug: 109936783
Test: atest DecoderTestXheAac ; atest DecoderTestAacDrc

Prevent bit buffer counter overflow.

While long-term test we discovered a bit counter overflow in the bit buffer.
The bit buffer state was only used by HCR and RVLC tool and can easily be substituted with FDKgetValidBits() call.
The following patch completely removes the bit counter and all its obsolete functions.

Change-Id: Icee0519d26a2aa62367d2dece59cd3d60ffcade7

Fix call to function sbrDecoder_Header() through pointer to incorrect function type.

While testing FDK in Android P DP3 x86 64bit emulator we observed a runtime error.
The function parameter of the callback didn't use a typeless pointer.
Following patch contains a revised sbr decoder callback function with explicit data type cast to be inline with all other callbacks in FDK.

Change-Id: Id224de22beaf2eb9631babd838c4c3ceae0a801e

Fix huffman decoder escape sequence length limitation.

Change-Id: Ib05cc2c065739c27b9720a24f90d0ce4d15bf601

Prevent overflow in MPEG-D DRC _setSelectionDataInfo().

Change-Id: I3b90ef7a5168e20766a2574d57d81bb39d2a5fb9

Add sanity check for allowed LPD sampling rate.

Change-Id: I0702024adc07ed463db795dd48f4efd3de1d7896

Always check whether given channel config is supported.

Change-Id: I169161dd31bc624f2cab6be2b4c6518946ed32ba

Unsigned Integer Overflow in aacDecoder_DecodeFrame().

Change-Id: I617dd545f9ea0aca5a5e7d214bbc35f089bc355d

Unsigned Integer Overflow in SpatialDecParseSpecificConfigHeader().

Change-Id: I5994a55f993835fa511ff61a337726b3e51aed5d

Unsigned Integer Overflows in mpegSurroundDecoder_Parse().

Change-Id: I3714e88d7a9a866297bb2e93571804665a97773f

Unsigned Integer Overflow in transportDec_AdjustEndOfAccessUnit().

Change-Id: Ie4924460d30ea4ef76917704ded809d04db78993

Unsigned Integer Overflow in transportDec_readHeader().

Change-Id: I0beedab38175fc57e5bd9eb5700a3850ef2bebf7

UBSAN overflow in FFT

An overflow in the fft was observed for certain configurations. The root cause was in the scaling of the fft input data in dct_IV and dst_IV.
The following patch revises the scaling in dct and removes the unused arm implementation.

Change-Id: Idc9c44a9755b8151d5c7d8107696f43ac592413b

1 files changed, 16 insertions, 13 deletions

diff --git a/libAACdec/src/rvlc.cpp b/libAACdec/src/rvlc.cpp
index 92f9f02..b7a9be1 100644
--- a/libAACdec/src/rvlc.cpp
+++ b/libAACdec/src/rvlc.cpp
@@ -168,13 +168,14 @@ static void rvlcInit(CErRvlcInfo *pRvlc,
   /* set base bitstream ptr to the RVL-coded part (start of RVLC data (ESC 2))
    */
   FDKsyncCache(bs);
+  pRvlc->bsAnchor = (INT)FDKgetValidBits(bs);
 
-  pRvlc->bitstreamIndexRvlFwd = FDKgetBitCnt(
-      bs); /* first bit within RVL coded block as start address for  forward
-              decoding */
-  pRvlc->bitstreamIndexRvlBwd = FDKgetBitCnt(bs) + pRvlc->length_of_rvlc_sf -
-                                1; /* last bit within RVL coded block as start
-                                      address for backward decoding */
+  pRvlc->bitstreamIndexRvlFwd =
+      0; /* first bit within RVL coded block as start address for  forward
+            decoding */
+  pRvlc->bitstreamIndexRvlBwd =
+      pRvlc->length_of_rvlc_sf - 1; /* last bit within RVL coded block as start
+                                       address for backward decoding */
 
   /* skip RVLC-bitstream-part -- pointing now to escapes (if present) or to TNS
    * data (if present) */
@@ -183,7 +184,7 @@ static void rvlcInit(CErRvlcInfo *pRvlc,
   if (pRvlc->sf_escapes_present != 0) {
     /* locate internal bitstream ptr at escapes (which is the second part) */
     FDKsyncCache(bs);
-    pRvlc->bitstreamIndexEsc = FDKgetBitCnt(bs);
+    pRvlc->bitstreamIndexEsc = pRvlc->bsAnchor - (INT)FDKgetValidBits(bs);
 
     /* skip escapeRVLC-bitstream-part -- pointing to TNS data (if present)   to
      * make decoder continue */
@@ -259,8 +260,9 @@ static SCHAR rvlcDecodeEscapeWord(CErRvlcInfo *pRvlc, HANDLE_FDK_BITSTREAM bs) {
   treeNode = *pEscTree; /* init at starting node */
 
   for (i = MAX_LEN_RVLC_ESCAPE_WORD - 1; i >= 0; i--) {
-    carryBit = rvlcReadBitFromBitstream(bs, /* get next bit */
-                                        pBitstreamIndexEsc, FWD);
+    carryBit =
+        rvlcReadBitFromBitstream(bs, /* get next bit */
+                                 pRvlc->bsAnchor, pBitstreamIndexEsc, FWD);
 
     CarryBitToBranchValue(carryBit, /* huffman decoding, do a single step in
                                        huffman decoding tree */
@@ -370,8 +372,9 @@ SCHAR decodeRVLCodeword(HANDLE_FDK_BITSTREAM bs, CErRvlcInfo *pRvlc) {
   UINT treeNode = *pRvlCodeTree;
 
   for (i = MAX_LEN_RVLC_CODE_WORD - 1; i >= 0; i--) {
-    carryBit = rvlcReadBitFromBitstream(bs, /* get next bit */
-                                        pBitstrIndxRvl, direction);
+    carryBit =
+        rvlcReadBitFromBitstream(bs, /* get next bit */
+                                 pRvlc->bsAnchor, pBitstrIndxRvl, direction);
 
     CarryBitToBranchValue(carryBit, /* huffman decoding, do a single step in
                                        huffman decoding tree */
@@ -1140,7 +1143,7 @@ void CRvlc_Decode(CAacDecoderChannelInfo *pAacDecoderChannelInfo,
   rvlcInit(pRvlc, pAacDecoderChannelInfo, bs);
 
   /* save bitstream position */
-  saveBitCnt = FDKgetBitCnt(bs);
+  saveBitCnt = (INT)FDKgetValidBits(bs);
 
   if (pRvlc->sf_escapes_present)
     rvlcDecodeEscapes(
@@ -1155,7 +1158,7 @@ void CRvlc_Decode(CAacDecoderChannelInfo *pAacDecoderChannelInfo,
   pAacDecoderChannelInfo->data.aac.PnsData.PnsActive = pRvlc->noise_used;
 
   /* restore bitstream position */
-  bitCntOffst = saveBitCnt - FDKgetBitCnt(bs);
+  bitCntOffst = (INT)FDKgetValidBits(bs) - saveBitCnt;
   if (bitCntOffst) {
     FDKpushBiDirectional(bs, bitCntOffst);
   }