Changing the box colour should be possible by defining a resource bundle and then using in a custom view template. I recommend starting by reading the OrgChart Admin Guide, section 5.10 called Resource Bundle Configuration.
Border lines (dotted/solid) are already used for dotted line functionality and I don't know how to change this. Shape I also don't know how/if it is possible to change. I have never attempted either of these last two items.
Stephen