Got some building work done to rewire my house, external office, and garage. When I spoke to the builder, he gave me a quote but said he needed to check with his boss. The boss said it would be more expensive but we were happy to pay this to get the work done. When the work is finally "finished", the external office and garage haven't been re-wired, and there are massive holes in the walls that haven't been plastered in. He's still demanding his full payment, saying that this is what we agreed to. We definitely didn't agree to this and always maintained we wanted the external buildings done too.
My husband has pretty bad memory issues after an accident a few years back, so doesn't remember the conversation accepting not to have this extra work included He says he never would have accepted this as we both wanted the external buildings done and made this clear throughout. There is a written quote, but we did not read it as we assumed it would be the same as all of our original agreements including the external building re-wiring. We would never have agreed to pay so much if we knew this work wouldn't be done. Obviously this is a lesson learned, but it was otherwise never made expressly clear there would be any changes to the original agreement.

I've tried to email them to sort this out and they're accusing me of trying to delay/withhold payment, and saying that they were abusing my husband for his memory issues. This is not the case, I just want to get the original work done for the original fee. I cannot accept that I will have to pay more money for less work! It just doesn't seem right. What should I do? Is small claims court the way to go? Do I have a case?