Publication Date:
2019-07-13
Description:
Improved method of rounding off (truncation of least-significant bits) in integer processing of data devised. Provides for reduction, to extremely low value, of numerical bias otherwise generated by accumulation of truncation errors from many arithmetic operations. Devised for use in integer signal processing, in which rescaling and truncation usually performed to reduce number of bits, which typically builds up in sequence of operations. Essence of method to alternate direction of roundoff (plus, then minus) on alternate occurrences of truncated values contributing to bias.
Keywords:
MATHEMATICS AND INFORMATION SCIENCES
Type:
NPO-18968
,
NASA Tech Briefs (ISSN 0145-319X); 19; 2; P. 80
Format:
text
Permalink