How old is the property itself? Is it from 1981, or is it rather earlier?
What your solicitor/agent are telling you is true - it's quite common - but it doesn't mean it's right, and won't be a problem. If the "only issue" is lack of actual correct plans, that's not of itself a problem - the issue is, has what has been done/changed been subject to relevant permissions?
Firstly, ignore (politely) the advice of anyone with a vested interest in the sale (agents, seller, etc), and make sure your solicitor is not one provided/recommended by the agent. (I may be reading your message wrongly, but it sounds like the solicitor is more relaxed about this than I'd expect them to be...) Then, get your solicitor to investigate - this is a basic element of their work for you - with the registos (in terms of land type, building licensing, etc) as well as the câmara - they should know what to do, in any case!
In practice, you might be able to buy it, live there, and have no issues - but you cannot rely on that, a change of approach by the local council could see enforcement, and that'll be against the owner at the time (which could be you!). Additionally, if you need licensing for anything (Alojamento Local, for instance - which is often more subject to inspection/licensing than longer term rental) or if you yourselves want to apply for permission for something, it could then be flushed out, and be a real headache. Or it could be an issue for a future buyer, when you come to sell.
If any aspects of the buildings are "illegal" (ie not approved, if they could be subject to it) you should use your solicitor to oblige the seller to sort this out, as in most cases, it's easier for the seller than the new owner to do this. If they sellers won't. this may be because they know what a headache it'll be for them... so proceed with caution.