Sam did come to wait for Mary.

He had been paying attention to Mary's action.

Sam had been depressed for a long time because the painting was returned last time.

He had thought that Mary accepted his gift, so he had been very happy. Who knew that Mary had given the painting back