How to soften and remove hardened, dry glue off a wall.?

We're doing some remodeling to a new house and the previous owners had a big mirror mounted above the fireplace. This mirror was attached to a big piece of cardboard which in turn was glued to the wall. We've removed everything but the hardened glue is still on the wall. What is the easiest way to soften this to remove it???