Thats why i removed the mounted stem and changed the handlebar too.Going down yes, but uphill it would be incredibly difficult to keep the front down. Its one of the negative sides to the SL, its not the best climber.
If you want to upgrade your fork, change the stem into 0 degrees one and maybe a handlebar with 20mm rise. Specialized delivers the SL with a 6 degree stem and a handlebar with 28mm rise. That´s useless, even when you leave the 150mm fork.