gstutils: Fix linear regression comparision
authorEdward Hervey <edward@centricular.com>
Fri, 24 Nov 2017 11:05:26 +0000 (12:05 +0100)
committerEdward Hervey <bilboed@bilboed.com>
Fri, 24 Nov 2017 14:40:14 +0000 (15:40 +0100)
commit741ff6a3710f8b56440fcc86b37369853ae051fd
tree40252e3942afa4ab8d42614836822b7dadecd5a8
parent3afc5750627ad47187abcff908952a5c859c4da4
gstutils: Fix linear regression comparision

The check for dropping precision was wrong when sxx and syy were negative.

if they are negative then "G_MAXINT64 - val" would always overflow

The check was meant to use G_MININT64 (like in the loop contained just
after).
gst/gstutils.c