"Negroes who have been so long inconvenienced and denied opportunities for development are naturally afraid of anything that sounds like discrimination."