The performance hit would be negligible.
It took 5 hours because I had some 50 custom girls with multiple image sets for each, and I had to rename the files when moving them, finding the next available number. So set 1 had 10 anal pictures, set 2 had 20, I'd have to rename all of the anal pictures in set 1 to Anal21, Anal22, etc and then move them, and then look for duplicates, and then renumber again. I don't know if the last renumbering step was needed or not, but since I don't have access to the source code I can't check.
Just dumping them all into an Anal directory would make it much, much faster to merge picture sets.