@mondrian Unfortunately, the sketch didn't come through, but in any event, there are limits to what a forum can tell you when it comes to legalites: many factors may come into play here: land categorisation, licensing, ownership, useage, etc.
The código civil sets out certain aspects of legal expectations in relation to "muros de meação" (somewhat the equivalent of party walls - but do not expect find direct equivalence). Other laws and regulations will come into play, and might on occasion contradict - you're going to need legal advice, I think.
You mention that the walls are up to the boundary (the outer face being on the line), which might imply they are not in themselves the boundary, and so not party walls - so it may be necessary to establish whether they are your sole responsibility, or a joint one. This already sounds like something to speak to a lawyer about - perhaps the one who helped you purchase the place initially.
In many circumstances, the law may well support your careful and limited access for maintenance, but not necessarily of absolute right (in other words, without prejudice to the rights of the other parties, who might reasonably be able to refuse). You are unlikely to be able to do more than is unavoidable on the other property, and will most likely not be able to require the removal of anything not on your property that is inconvenient or undesirable, such as the asbestos roof.
In practice, therefore, working with the neighbours in a quid pro quo basis to ensure you get as much of what you need as possible, with them saving face and not losing financially or in other respects, will be necessary. This can be extremely hard if you have a neighbour who will not cooperate, there's no nice way to dress that up. Privacy and individual rights are prized, and often seen to extend further than they do in reality. Enforcement against a neighbour can be expensive and time-consuming - local authorities and police generally shy away from neighbour conflict, so you'd be likely to need legal advice to take legal action in respect of any evidenced breach.
Meanwhile, I'd recommend transparency in talking in a relaxed way in the common interest with the neighbours, perhaps starting with whomever seems most well-disposed to help, perhaps gaining their support in dealing with the most intractable. It may not work, but it'll avoid accusations of unilateral action.