It is important to remember that this is not describing a literal event, but is rather the young woman expressing how she experienced her husband seeking intimacy.
Without explanation the man is suddenly gone, probably because of her initial negative response to his advances. The poem is making a statement about those times in intimate relationships when only one partner is ready for physical love.1
The moment is lost. It is an expression of deep and mutual sexual desire and yet about misunderstanding, loss, and separation.2
6 I opened to my beloved, but my beloved had turned and gone. My soul failed me when he spoke. I sought him, but found him not; I called him, but he gave no answer.