In the original Hebrew, the phrase is literally, in the morning, in the morning.
The combination occurs twelve times (see Isaiah 28:19). It points to ongoing action, a disciplined, continued, earnest engagement from God. The Lord worked this way in order that we may not think that he is liable to sudden impulses like men, to cast off or quickly forget those whom he has once undertaken to guard, whom he continues, on the contrary, to make the objects of his grace till the end, and never leaves destitute of consolation.
1
4 The Lord GOD has given me the tongue of those who are taught, that I may know how to sustain with a word him who is weary. Morning by morning he awakens; he awakens my ear to hear as those who are taught.