The report by Inspector General Russell George praised the IRS for responding to a 2006 complaint by Nina Olson, the national taxpayer advocate, that the agency had frozen refunds for thousands of taxpayers without notifying them or giving them a chance to challenge the action.

However, the audit said the agency's response in altering its 30-year-old Questionable Refund Program may have gone too far, USA Today reported Thursday.

Among the problems, the audit said recent changes "could negatively affect tax administration by not holding perpetrators of smaller-valued (fraud) schemes accountable."

It also said $15.9 million in refunds were made as a result of the softer enforcement because initial reviews of questionable claims were not completed within "a certain number of days."

IRS Criminal Investigation Chief Eileen Mayer told the newspaper the agency is studying the recommendations and is trying to balance taxpayer rights with proper enforcement.