26-year-old Lavery, who became Walsall's fifteenth signing of the summer earlier this month, was among a trio of players to take to the field shortly after the Saddlers had fallen two goals behind just seven minutes into the second-half.
Caolan would go on to make the most of his opportunity as he netted with his first touch, from the penalty spot, before grabbing another goal nineteen-minutes from time, cutting the deficit as his side looked to get back on level terms.
Despite securing his first goals in a Saddlers shirt Lavery admitted he was disappointed as he and his teammates failed to get back level and ultimately exited the competition at the first time of asking.
Speaking via the club's official website, Caolan said; "It was a disappointed dressing room after the defeat. We got ourselves a life line but then we switched off again and conceded a sloppy goal and we were on the back foot again.
"Then we managed to get ourselves back in the game again but unfortunately it just wasn’t meant to be. The lads are disappointed with the defeat but I think the most disappointing thing is that we got ourselves into that position in the first place.
"We’re going to go back to the training ground this week, watch the match back and analyse where we went wrong and then work on everything to make it right. I can see a lot of positives in this team though.
"It’s always nice as a striker to get on the score sheet early on but it’s a bitter sweet feeling. It’s nice to get on the score sheet like I say but it would have been even better if we had got through to the next round."
And after his cup brace, Red Deer, Alberta-born Lavery is hoping that he has given manager Darrell Clarke a selection headache heading into this weekend’s League Two clash with Crewe Alexandra.
Caolan continued; "Hopefully I have given the manager something to think about after the two goals I scored against Crawley. At the end of the day it is his decision that is final and we are all backing his decision.
"It’s one of those that I am happy to play wherever he wants me to play, whenever he wants me to play."