"We have just rewarded very bad and dangerous behavior," he said, saying Iran saw the deal as license to continue to enrich uranium.
"We've given them just enough breathing room -- the one thing that brought them to the table," he said. "We may have just encouraged more violence in the future than we've stopped."
The agreement struck between Iran and six world powers calls for Iran to slow its nuclear activities for six months, in exchange for relief from some sanctions, while a more permanent deal is worked out.
In response to Secretary of State John Kerry's call for those who opposed the deal to provide alternatives, Rogers suggested tougher sanctions against Iran.
"Finally, we were getting to the place that was actually going to really bite, and they were going to have to make the determination: is a nuclear weapon worth the destruction of our country financially?" he said. "What they have just done is stopped the second round" of sanctions.
Rogers also said the negotiations ignored historical precedent in other countries that have tested nuclear weapons.
"We made this mistake in Pakistan, we made this mistake in North Korea," he said. "History is a great judge here and a great teacher. Why would you make the same mistake to a nation that will proliferate a nuclear arms race in the Middle East if they're successful getting a nuclear weapon?"
Former CIA director Michael Hayden, also appearing on CNN, said that while "practically the worst of all possible outcomes" would be if the temporary agreement becomes permanent, it's too early to determine what will happen.
"We might be overreacting a bit both in overselling the deal and perhaps over-criticizing the deal. This is at best an interim agreement," he said. "I think we're really going to find out whether this is important in the next six months, not right now."