r3.in.v5d: Prevent integer overflow by changing literal constant type…

… on Cray (#4363) When the code is being compiled for CRAY HPC machines, a macro and a function to convert IEEE single precision floating point number to CRAY number are defined. To adjust the base, '16258' constant is being used, which according to C rules (C99, section 6.4.4.1, subsection semantics) fits into an integer. Right shifting that integer, which is of 32 bits, by 48 results in integer overflow. Avoid this by defining the literal constant with the long data type. This comes with an extended discussion in PR #4363 and an idea to remove the code completely. Signed-off-by: Mohan Yelugoti <[email protected]>
OSGeo · Oct 15, 2024 · ec2bc8a · ec2bc8a
1 parent f22c9a2
commit ec2bc8a
Showing 1 changed file with 2 additions and 2 deletions.
diff --git a/raster3d/r3.in.v5d/binio.c b/raster3d/r3.in.v5d/binio.c
@@ -147,7 +147,7 @@ static void if_to_c(long *t, const long *f)
 {
     if (*f != 0) {
         *t = (((*f & 0x8000000000000000) |
-               ((*f & 0x7f80000000000000) >> 7) + (16258 << 48)) |
+               ((*f & 0x7f80000000000000) >> 7) + (16258L << 48)) |
               (((*f & 0x007fffff00000000) >> 8) | (0x0000800000000000)));
         if ((*f << 1) == 0)
             *t = 0;
@@ -160,7 +160,7 @@ static void if_to_c(long *t, const long *f)
 #define IF_TO_C(T, F)                                                   \
     if (F != 0) {                                                       \
         T = (((F & 0x8000000000000000) |                                \
-              ((F & 0x7f80000000000000) >> 7) + (16258 << 48)) |        \
+              ((F & 0x7f80000000000000) >> 7) + (16258L << 48)) |       \
              (((F & 0x007fffff00000000) >> 8) | (0x0000800000000000))); \
         if ((F << 1) == 0)                                              \
             T = 0;                                                      \