I can't do Follow the Leader on any difficulty. Apart from that I can mostly get golds in Story Mode. I however just cheat on the stupid follow the leader.
But it is a bit hard for new users, and too easy for experienced ones. I think the speed you travel at and kart difficulty should be independent though. Also, locking all the tracks is just annoying. It means some players can't play most tracks, which is less fun.
And although this isn't exactly an in-game difficulty thing, karts with different weights and stuff should be abolished. It's too confusing. Back to the good old days!