Filenames in Darwin are UTF-8, so I wondered what would happen if you use a nonsensical sequence. It seems that you can make weird disappearing files, which show up in the Finder when you first open the folder they're in, and then quickly disappear. Sometimes they don't actually disappear until you try and click on them, which is tantalising.
If you have an OS X machine try this:
perl -e 'mkdir("foo\xED\xA0\x80bar") or die $!'