The head of Internal Revenue Service's (IRS) head and the union for its workers agree that the agency has struggled to do its job due to five years of budget cuts.

Since fiscal 2011, Congress has cut the IRS budget by a total of $1.2 billion to the current $10.9 billion. According to the National Treasury Employees Union, this led to a four-year hiring freeze, which cut the workforce from more than 108,000 workers to about 90,000. The union said that in New Jersey, about 980 people work for the IRS, down more than 21% from 2011.

In a speech last week before the Tax Policy Center, a joint project of the Urban Institute and the Brookings Institution, IRS Commissioner John Koskinen said, "I believe that the underfunding of the agency is the most critical challenge facing the IRS today. This is not just about the agency. It's about the entire tax system".

Colleen M. Kelley, president of the National Treasury Employees Union said that ill-advised cuts have hurt the economy and have affected taxpayers. Implementation of the world's most complex tax code is a huge responsibility and the IRS requires an appropriate level of staffing to do it in a proper manner.

Koskinen and the union mentioned that IRS employees have been able to answer fewer than 40% of the taxpayer calls the agency receives. It struggles to fight identity theft, in which criminals claim tax refunds under stolen identities.

Koskinen also added that a loss of auditors and enforcement employees has resulted in $7 billion to $8 billion a year in lost revenues. The Tax Foundation has questioned the claim that the IRS needs more funding.

Koskinen also referred to the growing complexity of the tax code in his speech. He pointed out that even while its budget has been cut, the IRS has been asked to handle more tax provisions, including the tax-related parts of the Affordable Care Act.