I think you will be fine with 32mm tires
I use 32 Gravel King tires on my cyclocross bike to ride gravel events in NY.
Farmer's Daughter Gravel Grinder in Chadam, NY coming up in a week. I rode it before with 32-35 mm tires and it was fine. I did not ride the NJ gravel grinder but I assume the roads are similar. I also have a road bike with 32 mm GP5000 slick tires. That bike will also be fine on the NY, NJ "gravel" roads, the only difference is the gearing, road gearing may not be enough to climb some dirt roads with 15 % grade. Had to modify my cross bike gearing to climb some steep and long dirt roads in NY.