Solving QCA6174 WiFi crashing, high ping, unusable on Linux

One glaring issue I have with my laptop (using a Qualcomm NIC) is the ridiculous high pings while using WiFi. > 400ms for some packets to reach a near by machine and back is totally unacceptable. It makes ssh unusably slow and even mosh on the edge of barely usable. The situation is so bad...