ArbyChess is a chess engine that allows arbitrary boards and pieces.
http://sourceforge.net/projects/arbychess/