Nancy Pelosi flicked her hand like she was shooing away a gnat.

She had just been asked whether she was concerned about her new challenger in next year’s election to represent San Francisco in Congress. His name, Saikat Chakrabarti, is unfamiliar to many of the city’s voters. But it’s one that gets under Ms. Pelosi’s skin, given the tense history between the two Democrats going back to his brief and contentious time on Capitol Hill.