Most SEOs know that page speed is a Google ranking factor.But did you know that Google only measures the speed of your “Desktop site”?
The speed of your mobile site does not matter at all, at least not for Google.
This will change when the mobile-first index gets introduced, probably be in 2018.