Leave It Open:
The 'backwards' lyrics at the end of the song aren't really backwards. They've been recorded backwards, but they are pretty much straight forward and say "we let the wierdness in". Kate listened to the phrase backwards and tried to sing what she heard, which then was played backwards again - resulting in a wierd sounding forward phrase. On the other hand, you can hear something backwards as well: "And they said they won't let me in" (this isn't official, while the forward phrase is - it was a long unanswered quiz question in the KBC).
Harry Houdini (born Erich Weiss) was an escapologist who managed to escape by using a key that was passed to him by his wife Bess with a kiss she always gave him before he was bound and drowned. ("With a kiss, I'd pass the key.") Now Houdini and his wife previously agreed on a secret code ("Rosabel, believe!"), which he should use via a medium after his death to let her know about after-life. At one point Bess was given the code by a "psychic," but she later found out that he obtained it from a former servant of Houdini's (it *was* simply their favorite song, rather than some involved code.) So, it's easy to see how the confusion has happened.
Bess *once* stated that a "psychic" had given her the code, then she retracted that statement. As usually happens, the people who want to believe in "psychic phenomena" accept want they want and disregard the rest. So the first claim of contact is more widely reported than the true (but less romantic) story. Kate's song is closer in the details to the Hollywood film starring Tony Curtis than to the real events of Houdini's life and art. This is in no way intended to disparage Kate's song, which concentrates on the *emotional*, rather than the biographical aspects of Houdini and Bess' relationship. [some of this by CW]