I think you have just gone beyond the geometry what Specialized designed the Levo to work....I had the Gen 2 Levo also and I'm not heavy either. I had problems with front end traction and I lowered the stem 10mm and problem was solved...Wrong bike, wrong math. Her's was a 2019 Turbo Levo, which stock had a 66d head angle with 29" & 150mm F&R. Going 27.5" F&R dropped both ends equally, we added 30mm to the fork length which knocked 1.5d off the head angle. BB landed at 13.25". And she is 174cm tall, on a M frame.
Levo has quite high stack (or not low )so if you have even higher stack with 170mm fork, maybe it's just too high?