Yes, it would in many cases be the recruitment agency - however working practices normally trump whats in the contract so the end client would need to be involved too.
I can see one of two things happening:
1. Agents play it safe and everyone is inside...
1a. Rates go up (As happened with public sector)
1b. Contractors leave and clients struggle to complete projects (as happened with public sector)
1c. Contractors go back to perm work (See 1b - but also HMRC take decreases because permies dont gross as much)
2. Clients and agents change contracts and working practices so that contractors are not inside IR35... Nothing much changes.
Its probably going to end up somewhere in the middle of the two...