It's completely unfair. Ye is being accused of doping because her performance was unexpected by people who haven't been following her.
People tend to like predictability. Breaking a 400 meter record by one second is much less of an abnormality than winning eight gold medals. But since American media repeated over and over how phenomenal Michael Phelps was, there was no surprise involved.
People also like having a reason for why events - especially unpredictable ones - happened. To many people, including people deeply involved in swimming, Ye is an unknown. How can an unknown person break a world record? The only explanation at hand is performance-enhancing drugs.
Here are some facts that will help explain why Ye's performance should not have been as much of a surprise as it was:
One is that the bar of "better final 50 split than Ryan Lochte in the same event" is hardly insurmountable. Lochte had a 3-4 second lead going into that final 50 and, with a dozen events left to swim in the week, was justifiably on cruise control. No doubt if Phelps had been breathing down his neck, he would have been going faster. Lochte also doesn't have the fastest final 50 in the 400 IM - at Olympic Trials, he was 13th of 109 (with several men within a tenth of a second) in prelims and 4th of 8 in the finals (placing first overall in both). Is Ye's freestyle split ridiculous and unprecedented? Yes. But the comparison to Lochte, despite his being one of the best in the world at finishing a 400 IM, is a false call to authority.
The other is that Ye actually has had a reputation for blowing away her opponents in the freestyle leg of the IM for over a year. See this article from last July: http://theswimmerscircle.
The improvement in her performance also isn't as sudden as it might appear. She put up a ~4:35 in July 2011, a 4:33 in October (http://swimswam.com/2011
This all isn't to say definitively one way or the other if she's doping - only that her performance in the 400 IM at the Olympics is not reason enough to accuse her. She should receive the same scrutiny as if she finished second.