Fix overflow issues in image rotation
Also, do some refactoring to simplify code. * src/dispextern.h (INIT_MATRIX, COPY_MATRIX, MULT_MATRICES): Remove. * src/image.c (matrix3x3): New type, replacing all uses of 3x3 double. (matrix3x3_copy, matrix3x3_mult): New functions, replacing COPY_MATRIX, MULT_MATRICES. Replace INIT_MATRIX by C initializers. (image_set_rotation): Use Fmod to avoid undefined behavior on double-to-int conversion and to reduce bignum rotations correctly. (image_set_crop): Finish up previous correction, by not re-setting width and height if compute_image_size has set them. Prefer shifting right by 1 to dividing by 2 if either will do.
Showing with 77 additions and 93 deletions