FDK sanitization fixes

Bug: 110641440
Bug: 109936783
Test: atest DecoderTestXheAac ; atest DecoderTestAacDrc

Prevent bit buffer counter overflow.

While long-term test we discovered a bit counter overflow in the bit buffer.
The bit buffer state was only used by HCR and RVLC tool and can easily be substituted with FDKgetValidBits() call.
The following patch completely removes the bit counter and all its obsolete functions.

Change-Id: Icee0519d26a2aa62367d2dece59cd3d60ffcade7

Fix call to function sbrDecoder_Header() through pointer to incorrect function type.

While testing FDK in Android P DP3 x86 64bit emulator we observed a runtime error.
The function parameter of the callback didn't use a typeless pointer.
Following patch contains a revised sbr decoder callback function with explicit data type cast to be inline with all other callbacks in FDK.

Change-Id: Id224de22beaf2eb9631babd838c4c3ceae0a801e

Fix huffman decoder escape sequence length limitation.

Change-Id: Ib05cc2c065739c27b9720a24f90d0ce4d15bf601

Prevent overflow in MPEG-D DRC _setSelectionDataInfo().

Change-Id: I3b90ef7a5168e20766a2574d57d81bb39d2a5fb9

Add sanity check for allowed LPD sampling rate.

Change-Id: I0702024adc07ed463db795dd48f4efd3de1d7896

Always check whether given channel config is supported.

Change-Id: I169161dd31bc624f2cab6be2b4c6518946ed32ba

Unsigned Integer Overflow in aacDecoder_DecodeFrame().

Change-Id: I617dd545f9ea0aca5a5e7d214bbc35f089bc355d

Unsigned Integer Overflow in SpatialDecParseSpecificConfigHeader().

Change-Id: I5994a55f993835fa511ff61a337726b3e51aed5d

Unsigned Integer Overflows in mpegSurroundDecoder_Parse().

Change-Id: I3714e88d7a9a866297bb2e93571804665a97773f

Unsigned Integer Overflow in transportDec_AdjustEndOfAccessUnit().

Change-Id: Ie4924460d30ea4ef76917704ded809d04db78993

Unsigned Integer Overflow in transportDec_readHeader().

Change-Id: I0beedab38175fc57e5bd9eb5700a3850ef2bebf7

UBSAN overflow in FFT

An overflow in the fft was observed for certain configurations. The root cause was in the scaling of the fft input data in dct_IV and dst_IV.
The following patch revises the scaling in dct and removes the unused arm implementation.

Change-Id: Idc9c44a9755b8151d5c7d8107696f43ac592413b

2 files changed, 0 insertions, 52 deletions

diff --git a/libFDK/include/FDK_bitbuffer.h b/libFDK/include/FDK_bitbuffer.h
index ed0b2f6..19a24b3 100644
--- a/libFDK/include/FDK_bitbuffer.h
+++ b/libFDK/include/FDK_bitbuffer.h
@@ -113,7 +113,6 @@ typedef struct {
   UINT ValidBits;
   UINT ReadOffset;
   UINT WriteOffset;
-  UINT BitCnt;
   UINT BitNdx;
 
   UCHAR *Buffer;
@@ -159,15 +158,10 @@ void FDK_pushBack(HANDLE_FDK_BITBUF hBitBuffer, const UINT numberOfBits,
 void FDK_pushForward(HANDLE_FDK_BITBUF hBitBuffer, const UINT numberOfBits,
                      UCHAR config);
 
-void FDK_byteAlign(HANDLE_FDK_BITBUF hBitBuffer, UCHAR config);
-
 UINT FDK_getValidBits(HANDLE_FDK_BITBUF hBitBuffer);
 
 INT FDK_getFreeBits(HANDLE_FDK_BITBUF hBitBuffer);
 
-void FDK_setBitCnt(HANDLE_FDK_BITBUF hBitBuffer, const UINT value);
-INT FDK_getBitCnt(HANDLE_FDK_BITBUF hBitBuffer);
-
 void FDK_Feed(HANDLE_FDK_BITBUF hBitBuffer, const UCHAR inputBuffer[],
               const UINT bufferSize, UINT *bytesValid);
 
diff --git a/libFDK/include/FDK_bitstream.h b/libFDK/include/FDK_bitstream.h
index 49eeeaf..f799026 100644
--- a/libFDK/include/FDK_bitstream.h
+++ b/libFDK/include/FDK_bitstream.h
@@ -481,21 +481,6 @@ FDK_INLINE void FDKsyncCacheBwd(HANDLE_FDK_BITSTREAM hBitStream) {
 }
 
 /**
- * \brief Byte Alignment Function.
- *        This function performs the byte_alignment() syntactic function on the
- * input stream, i.e. some bits will be discarded/padded so that the next bits
- * to be read/written will be aligned on a byte boundary with respect to
- * the bit position 0.
- *
- * \param  hBitStream HANDLE_FDK_BITSTREAM handle
- * \return void
- */
-FDK_INLINE void FDKbyteAlign(HANDLE_FDK_BITSTREAM hBitStream) {
-  FDKsyncCache(hBitStream);
-  FDK_byteAlign(&hBitStream->hBitBuf, (UCHAR)hBitStream->ConfigCache);
-}
-
-/**
  * \brief Byte Alignment Function with anchor
  *        This function performs the byte_alignment() syntactic function on the
  * input stream, i.e. some bits will be discarded so that the next bits to be
@@ -604,37 +589,6 @@ FDK_INLINE INT FDKgetFreeBits(HANDLE_FDK_BITSTREAM hBitStream) {
 }
 
 /**
- * \brief reset bitcounter in bitBuffer to zero.
- * \param hBitStream HANDLE_FDK_BITSTREAM handle
- * \return void
- */
-FDK_INLINE void FDKresetBitCnt(HANDLE_FDK_BITSTREAM hBitStream) {
-  FDKsyncCache(hBitStream);
-  FDK_setBitCnt(&hBitStream->hBitBuf, 0);
-}
-
-/**
- * \brief set bitcoutner in bitBuffer to given value.
- * \param hBitStream HANDLE_FDK_BITSTREAM handle
- * \param value new value to be assigned to the bit counter
- * \return void
- */
-FDK_INLINE void FDKsetBitCnt(HANDLE_FDK_BITSTREAM hBitStream, UINT value) {
-  FDKsyncCache(hBitStream);
-  FDK_setBitCnt(&hBitStream->hBitBuf, value);
-}
-
-/**
- * \brief get bitcounter state from bitBuffer.
- * \param hBitStream HANDLE_FDK_BITSTREAM handle
- * \return current bit counter value
- */
-FDK_INLINE INT FDKgetBitCnt(HANDLE_FDK_BITSTREAM hBitStream) {
-  FDKsyncCache(hBitStream);
-  return FDK_getBitCnt(&hBitStream->hBitBuf);
-}
-
-/**
  * \brief Fill the BitBuffer with a number of input bytes from  external source.
  *        The bytesValid variable returns the number of ramaining valid bytes in
  * extern inputBuffer.