He Remains Faithful

The last line of an ancient saying always puzzled me.