After suffering a run of three straight losses, the Saddlers returned to winning ways last weekend with Josh Gordon and Wes McDonald netting in a two-nil victory over Forest Green Rovers, but Clarke feels team continues to shoot itself in the foot.
Speaking via the Express & Star, Darrell said; "Don’t get me wrong, I’m a frustrated manager. But I’m not sat here frustrated because I’ve spent loads of money, I haven’t got the pressure of that.
"I have the pressure of, and I’ve had this at my previous clubs, is where the football club has been before. I mention these words all the time, I build a siege mentality because they’ll be a minority of fans that see where the club has been, but we know it’s a building job.
"We talk about the Walsall way and getting young lads through the system and I think we’ve done that this season. There’s a lot of positives in that and development stages.
"You have to bear in mind that before the previous week we were in top three form. We had a really bad week where we conceding some sloppy goals, which has been a bit of a theme at times this year.
"We’re building a new group and a new squad together without the resources that a lot of teams have at this level. We’re a building club and the fans aren’t daft, they appreciate that.
"I’m frustrated because I think we could have been further afield up there but we shot ourselves in the foot on too many occasions."