HD6432138SW datasheet(624/1061 Pages) RENESAS

624 / 1061 page

Section 19 A/D Converter

Rev. 4.00 Jun 06, 2006 page 570 of 1004

REJ09B0301-0400

A/D Conversion Precision Definitions: H8S/2138 Group and H8S/2134 Group A/D conversion

precision definitions are given below.

• Resolution

The number of A/D converter digital output codes

• Offset error

The deviation of the analog input voltage value from the ideal A/D conversion characteristic

when the digital output changes from the minimum voltage value B'0000000000 (H'000) to

B'0000000001 (H'001) (see figure 19.10).

• Full-scale error

The deviation of the analog input voltage value from the ideal A/D conversion characteristic

when the digital output changes from B'1111111110 (H'3FE) to B'111111111 (H'3FF) (see

figure 19.10).

• Quantization error

The deviation inherent in the A/D converter, given by 1/2 LSB (see figure 19.9).

• Nonlinearity error

The error with respect to the ideal A/D conversion characteristic between the zero voltage and

the full-scale voltage. Does not include the offset error, full-scale error, or quantization error.

• Absolute precision

The deviation between the digital value and the analog input value. Includes the offset error,

full-scale error, quantization error, and nonlinearity error.

HD6432138SW Datasheet(PDF) 624 Page - Renesas Technology Corp