diff options
| author | Jean-Michel Trivi <jmtrivi@google.com> | 2013-08-08 17:38:33 -0700 |
|---|---|---|
| committer | Jean-Michel Trivi <jmtrivi@google.com> | 2013-08-20 15:01:21 -0700 |
| commit | ed247dfa54234c1115c35a05327955b29b161e8a (patch) | |
| tree | 9e4488bf844e217bf16d8cd3f001738d14ed56fd /libAACenc/src/transform.h | |
| parent | 577fcbb570d023be4cea9564292dd2bd95f40c3b (diff) | |
| download | fdk-aac-dabplus-ed247dfa54234c1115c35a05327955b29b161e8a.tar.gz fdk-aac-dabplus-ed247dfa54234c1115c35a05327955b29b161e8a.tar.bz2 fdk-aac-dabplus-ed247dfa54234c1115c35a05327955b29b161e8a.zip | |
Encoder quantizer fix
AAC Encoder: Saturate quantizer shift value to prevent
undefined behaviour.
In very rare cases the shift value 'totalShift = (16-4)-(3*(totalShift>>2))'
can be greater than accu data width. If you apply a shift with more then
31 bit the result depends on the architecture and is not defined in C.
For certain platforms zeros are shifted in. That would be our desired behaviour.
On other platforms the shift will be applied as modulo. For example >>34
would be applied as >>2. To prevent this discrepancy the shift value
is limited/saturated to DFRACT_BITS-1. 'accu >>= fixMin(totalShift,DFRACT_BITS-1)'.
Bug 9428126
Change-Id: I27177654c4dc22cf899bc35dad9cdd040dccb02d
Diffstat (limited to 'libAACenc/src/transform.h')
0 files changed, 0 insertions, 0 deletions