I’m decades past my ability to provide the numbers to substantiate what follows, so this is just a generalization of what happens to air flow in a duct…
For a given volumetric flow rate (cubic feet per minute established by the roaster’s exhaust fan), air flow velocity is higher for a smaller diameter duct. It’s a direct relationship. The issue is any back pressure associated with reduced cross sectional area associated with drag caused by the surface of the duct (rough/uneven surface = drag; geometric shape can also induce drag). Any back pressure will reduce air velocity and as a result the volumetric flow. In general, lower air flow velocity in the exhaust duct will probably mean lower back pressure resulting in a smaller negative effect on volumetric flow. And it’s volumetric flow that you want to maximize.
So a rule of thumb would be to use the largest diameter duct you can fit with the smoothest possible interior surface, the fewest obstructions, and the fewest turns/angles/twists. In the extreme you could us a 10" sheet metal stove flue with as few joints as possible, but the improvement between that and say a 4" sheet metal flue would likely be so minimal it would be a waste of $$$.
Alternate to filling your roasting space with large diameter ducting is to insert an in-line fan in the duct, e.g. drier host ducting, to compensate for any duct-induced back pressure. If you choose this route you are at risk of affecting the thermal differences created by setting Fan speed. So a system that allows ambient air to mix with ducted air will cause the least change in flow rate inside the roaster for a given F#.