I don't mind the color system per say, I think it is an interesting mechanic, but in its current incarnation, there is no explanation anywhere is the game or associated documentation that tells you a) what it is you are doing when you assign the colors to slots, b) what is happening when you pick a color for a class, c) which classes and/or teachers prefer which colors, or d) what the result of choosing a color is. I've pretty much figured out which colors are good for which classes, but I still have no idea what I'm actually doing when I pick the color for a class.
If I pick red, am I emphasizing sex in the class? Am I playing subliminal messages over the speakers that make students horny? I have no clue.
If you keep the basic game mechanic that there are choices the player can make that have some influence over the class being taught (leads to stat changes in the students, teacher, or school and/or gives or takes away orbs), but change the way in which it is presented to the player (do away with putting colors in slots and picking them for classes), then you keep what is good about the system and are rid of the confusing, meaningless parts.